Monitoring a user of a head-wearable electronic device

ABSTRACT

Systems, methods, and computer-readable media for monitoring a user of a head-wearable electronic device with multiple light-sensing assemblies.

CROSS-REFERENCE(S) TO RELATED APPLICATION(S)

This application claims the benefit of prior filed U.S. ProvisionalPatent Application No. 62/565,272, filed Sep. 29, 2017, and prior filedU.S. Provisional Patent Application No. 62/718,937, filed Aug. 14, 2018,each of which is hereby incorporated by reference herein in itsentirety.

TECHNICAL FIELD

This disclosure relates to the monitoring of a user of a head-wearableelectronic device and, more particularly, to the monitoring of a user ofa head-wearable electronic device with multiple light-sensingassemblies.

BACKGROUND OF THE DISCLOSURE

A portable electronic device (e.g., a cellular telephone) may beprovided with one or more sensing components (e.g., one or more touchsensors, sound sensors, etc.) that may be utilized for enabling a userto control a functionality of the electronic device. However, suchcontrol often requires the user to interact with the sensing componentsactively, such as via touch or speech.

SUMMARY OF THE DISCLOSURE

This document describes systems, methods, and computer-readable mediafor monitoring a user of a head-wearable electronic device.

For example, a method of detecting head gestures performed by a user'shead wearing an electronic device including a plurality of light-sensingcomponents is provided that may include, during a first period in whichthe user performs a first head gesture, collecting first sensor datafrom the plurality of light-sensing components, during a second periodin which the user performs a second head gesture, collecting secondsensor data from the plurality of light-sensing components, calculatingfirst signal characteristics based on the first sensor data, calculatingsecond signal characteristics based on the second sensor data, assigningsome or all of the first signal characteristics to a first cluster ofsignal characteristics, assigning some or all of the second signalcharacteristics to a second cluster of signal characteristics, during athird period, collecting third sensor data from the plurality oflight-sensing components, calculating third signal characteristics basedon the third sensor data, determining whether the third signalcharacteristics belong to the first cluster, the second cluster, or athird cluster, in accordance with a determination that the third signalcharacteristics belong to the first cluster, determining that the userhas performed the first head gesture, in accordance with a determinationthat the third signal characteristics belong to the second cluster,determining that the user has performed the second head gesture, and, inaccordance with a determination that the third signal characteristicsbelong to the third cluster, determining that the user has not performedthe first head gesture or the second head gesture.

As another example, a method for monitoring a user wearing ahead-wearable electronic device (HWD) on the user's head using a headgesture model custodian system is provided that may include initiallyconfiguring, at the head gesture model custodian system, a learningengine, receiving, at the head gesture model custodian system from theHWD, HWD sensor category data for at least one HWD sensor category for ahead gesture and a score for the head gesture, training, at the headgesture model custodian system, the learning engine using the receivedHWD sensor category data and the received score, accessing, at the headgesture model custodian system, HWD sensor category data for the atleast one HWD sensor category for another head gesture, scoring theother head gesture, using the learning engine for the HWD at the headgesture model custodian system, with the accessed HWD sensor categorydata for the other head gesture, and, when the score for the other headgesture satisfies a condition, generating, with the head gesture modelcustodian system, control data associated with the satisfied condition.

As yet another example, a head-wearable electronic device is providedthat may include a head-wearable housing structure including an eyeframe, a right temple frame extending from the eye frame, and a lefttemple frame extending from the eye frame, wherein, when thehead-wearable electronic device is worn on a head of a user, thehead-wearable housing structure is configured such that the eye frame ispositioned in front of at least one eye of the user's head, the righttemple frame is held against a right surface of the user's head, and theleft temple frame is held against a left surface of the user's head, aright light-sensing assembly supported by the right temple frameincluding a right light-emitting component operative to emit light intothe right surface of the user's head when the head-wearable electronicdevice is worn on the head of the user, and a right light-sensingcomponent operative to sense right light including at least a portion ofthe light emitted by the right light-emitting component, a leftlight-sensing assembly supported by the left temple frame including aleft light-emitting component operative to emit light into the leftsurface of the user's head when the head-wearable electronic device isworn on the head of the user, and a left light-sensing componentoperative to sense left light including at least a portion of the lightemitted by the left light-emitting component, and a processor operativeto analyze light data indicative of at least one of the sensed rightlight and the sensed left light, and determine a head gesture of theuser based on the analyzed light data.

As yet another example, a product is provided that may include anon-transitory computer-readable medium and computer-readableinstructions, stored on the computer-readable medium, that, whenexecuted, are effective to cause a computer to receive, from ahead-wearable electronic device (HWD) worn by a user, sensor categorydata for at least one HWD sensor category for a head gesture of the userand a type of the head gesture, train a learning engine using thereceived HWD sensor category data and the received type of the headgesture, access HWD sensor category data for the at least one HWD sensorcategory for another head gesture of the user, and determine a type ofthe other head gesture, using the learning engine with the accessed HWDsensor category data for the other head gesture.

This Summary is provided only to summarize some example embodiments, soas to provide a basic understanding of some aspects of the subjectmatter described in this document. Accordingly, it will be appreciatedthat the features described in this Summary are only examples and shouldnot be construed to narrow the scope or spirit of the subject matterdescribed herein in any way. Unless otherwise stated, features describedin the context of one example may be combined or used with featuresdescribed in the context of one or more other examples. Other features,aspects, and advantages of the subject matter described herein willbecome apparent from the following Detailed Description, Figures, andClaims.

BRIEF DESCRIPTION OF THE DRAWINGS

The discussion below makes reference to the following drawings, in whichlike reference characters refer to like parts throughout, and in which:

FIG. 1 is a schematic view of an illustrative system with ahead-wearable electronic device for monitoring a user;

FIG. 2 shows an illustrative embodiment of the system of FIG. 1 in useby a user;

FIG. 2A shows a perspective view of the head-wearable electronic deviceof the system of FIGS. 1 and 2;

FIG. 2B shows a cross-sectional view of the head-wearable electronicdevice of the system of FIGS. 1-2A, taken from line IIB-IIB of FIG. 2;

FIG. 2C shows a portion of the head-wearable electronic device of thesystem of FIGS. 1-2B;

FIG. 2D shows a cross-sectional view of the head-wearable electronicdevice of the system of FIGS. 1-2C, taken from line IID-IID of FIG. 2B;

FIG. 2E shows a cross-sectional view of the head-wearable electronicdevice of the system of FIGS. 1-2D, taken from line of FIG. 2B;

FIG. 2F shows a cross-sectional view of the head-wearable electronicdevice of the system of FIGS. 1-2E, taken from line IIF-IIF of FIG. 2B;

FIG. 2G shows a cross-sectional view of the head-wearable electronicdevice of the system of FIGS. 1-2F, taken from line IIG-IIG of FIG. 2B;

FIG. 2H shows a cross-sectional view of the head-wearable electronicdevice of the system of FIGS. 1-2G, taken from line IIH-IIH of FIG. 2B;

FIG. 2I shows another illustrative embodiment of the system of FIG. 1 inuse by a user;

FIG. 3 is a schematic view of an illustrative portion of thehead-wearable electronic device of FIGS. 1-2H;

FIGS. 4A-4E illustrate exemplary head gestures in accordance withexamples of the disclosure;

FIG. 5 illustrates exemplary charts of sensor data in accordance withexamples of the disclosure;

FIGS. 5A-5D illustrate two-dimensional clustering examples in accordancewith examples of the disclosure; and

FIGS. 6-10 are flowcharts of illustrative processes for monitoring auser of a head-wearable electronic device.

DETAILED DESCRIPTION OF THE DISCLOSURE

Systems, methods, and computer-readable media may be provided to monitora user of a head-wearable electronic device. Such a head-wearableelectronic device may be any suitable structure that may be worn on anysuitable portion of a user's head, including, but not limited to,eyeglasses (e.g., a pair of augmented reality eyeglasses, readingglasses, sunglasses, etc.), virtual reality head-mounted display,goggles (e.g., athletic goggles, welding goggles, etc.), hat, helmet,headband, or the like, and that may provide one or more light sensingassemblies operative to detect light reflected by and/or transmittedthrough a portion of the user's head (e.g., using photoplethysmography(“PPG”)). For example, the head-wearable electronic device may includeany suitable number of photodiodes or any other suitable type(s) ofoptical sensors, each of which may be provided at a respective differentlocation along the structure of the head-wearable device in order tosense light at a respective different position along the head of theuser when the device is worn by the user's head. Due to suchpositioning, the sensor data from the light sensors can capture movementof anatomical features in the tissue of the head of the user and can beused to determine any suitable gestures of the user, where such usergestures or head gestures may refer to any suitable gestures (e.g.,voluntary and/or involuntary gestures of any suitable portion(s) of auser's head) and/or motions and/or actions and/or vocalizations and/oremotions and/or thoughts and/or brain functions and/or heart ratecharacteristics and/or other biometric characteristics of the user.Further, different light emitters of the device can emit light atdifferent wavelengths (e.g., infrared (“IR”) light, green light, etc.),which may penetrate to different depths in the tissue of the user's headbefore reflecting back to or otherwise being detected by the photodiodesof the device. Accordingly, sensor data from the photodiodes can captureexpansion, contraction, and/or any other suitable movement in the tissueof the user during a head gesture. Various different head gestures,including, but not limited to, chewing, blinking, winking, smiling,eyebrow raising, jaw motioning (e.g., jaw protrusion, jaw retrusion,lateral jaw excursion, jaw depression, jaw elevation, etc.), mouthopening, and/or the like, may be detected by recognizing patterns insensor data that may be characteristic of each head gesture, as headtissue may expand and contract and anatomical features in the tissue maymove during such user gestures.

FIG. 1 is a schematic view of an illustrative system 1 that includes ahead-wearable electronic device 100 for monitoring a user. Head-wearableelectronic device 100 may be any suitable electronic device that mayinclude at least one light-sensing assembly and that may be at leastpartially worn on any suitable portion of a user's head. For example,head-wearable electronic device 100 may include, but is not limited to,a helmet, eyeglasses (e.g., sunglasses, reading glasses, noveltyglasses, augmented reality eyeglasses, etc.), a headset, headphones,earphones, over-ear speakers, in-ear speakers, a head-mounted displaydevice (e.g., any monocular or binocular or optical head-mounted displaydevice for virtual reality applications or any other suitable use),goggles, a hat, a headband, a mask, a hood, a head chain, earrings,earmuffs, or any combination thereof that may be at least partially wornon a user's head and operative to position at least one light-sensingassembly against or facing or otherwise proximate any suitable portionof the user's head for sensing light that may be reflected therebyand/or transmitted therethrough.

As shown in FIG. 1, for example, head-wearable electronic device 100 mayinclude a processor assembly 102, a memory assembly 104, acommunications assembly 106, a power supply assembly 108, an inputassembly 110, an output assembly 112, and a sensor assembly 114.Head-wearable electronic device (“HWD”) 100 may also include a bus 116that may provide one or more wired or wireless communication links orpaths for transferring data and/or power to, from, or between variousassemblies of HWD 100. In some embodiments, one or more assemblies ofHWD 100 may be combined or omitted. Moreover, HWD 100 may include anyother suitable assemblies not combined or included in FIG. 1 and/orseveral instances of the assemblies shown in FIG. 1. For the sake ofsimplicity, only one of each of the assemblies is shown in FIG. 1.

Memory assembly 104 may include one or more storage mediums, including,for example, a hard-drive, flash memory, permanent memory such asread-only memory (“ROM”), semi-permanent memory such as random accessmemory (“RAM”), any other suitable type of storage assembly, or anycombination thereof. Memory assembly 104 may include cache memory, whichmay be one or more different types of memory used for temporarilystoring data for electronic device applications. Memory assembly 104 maybe fixedly embedded within electronic device 100 or may be incorporatedonto one or more suitable types of components that may be repeatedlyinserted into and removed from HWD 100 (e.g., a subscriber identitymodule (“SIM”) card or secure digital (“SD”) memory card). Memoryassembly 104 may store media data (e.g., music and image files),software (e.g., for implementing functions on HWD 100), firmware,preference information (e.g., media playback preferences), lifestyleinformation (e.g., food preferences), exercise information (e.g.,information obtained by exercise monitoring applications), sleepinformation (e.g., information obtained by sleep monitoringapplications), mindfulness information (e.g., information obtained bymindfulness monitoring applications), transaction information (e.g.,credit card information), wireless connection information (e.g.,information that may enable HWD 100 to establish a wireless connection),subscription information (e.g., information that keeps track of podcastsor television shows or other media a user subscribes to), contactinformation (e.g., telephone numbers and e-mail addresses), calendarinformation, pass information (e.g., transportation boarding passes,event tickets, coupons, store cards, financial payment cards, etc.), anysuitable head gesture data of HWD 100 (e.g., as may be stored in anysuitable head gesture cluster data 105 of memory assembly 104), anyother suitable data, or any combination thereof.

Communications assembly 106 may be provided to allow HWD 100 tocommunicate with one or more other electronic devices or servers orsubsystems or any other entities remote from HWD 100 (e.g., withelectronic subsystem 200 of system 1 of FIG. 1) using any suitablecommunications protocol(s). For example, communications assembly 106 maysupport Wi-Fi™ (e.g., an 802.11 protocol), ZigBee™ (e.g., an 802.15.4protocol), WiDi™, Ethernet, Bluetooth™, Bluetooth™ Low Energy (“BLE”),high frequency systems (e.g., 900 MHz, 2.4 GHz, and 5.6 GHzcommunication systems), infrared, transmission control protocol/internetprotocol (“TCP/IP”) (e.g., any of the protocols used in each of theTCP/IP layers), Stream Control Transmission Protocol (“SCTP”), DynamicHost Configuration Protocol (“DHCP”), hypertext transfer protocol(“HTTP”), BitTorrent™, file transfer protocol (“FTP”), real-timetransport protocol (“RTP”), real-time streaming protocol (“RTSP”),real-time control protocol (“RTCP”), Remote Audio Output Protocol(“RAOP”), Real Data Transport Protocol™ (“RDTP”), User Datagram Protocol(“UDP”), secure shell protocol (“SSH”), wireless distribution system(“WDS”) bridging, near field communication (“NFC”), any communicationsprotocol that may be used by wireless and cellular telephones andpersonal e-mail devices (e.g., Global System for Mobile Communications(“GSM”), GSM plus Enhanced Data rates for GSM Evolution (“EDGE”), CodeDivision Multiple Access (“CDMA”), Orthogonal Frequency-DivisionMultiple Access (“OFDMA”), high speed packet access (“HSPA”),multi-band, etc.), any communications protocol that may be used by a lowpower Wireless Personal Area Network (“6LoWPAN”) module, any othercommunications protocol, or any combination thereof. Communicationsassembly 106 may also include or may be electrically coupled to anysuitable transceiver circuitry that can enable HWD 100 to becommunicatively coupled to another device (e.g., a server, hostcomputer, scanner, accessory device, subsystem, etc.) and communicatedata with that other device wirelessly or via a wired connection (e.g.,using a connector port). Communications assembly 106 (and/or sensorassembly 114) may be configured to determine a geographical position ofHWD 100 and/or any suitable data that may be associated with thatposition. For example, communications assembly 106 may utilize a globalpositioning system (“GPS”) or a regional or site-wide positioning systemthat may use cell tower positioning technology or Wi-Fi™ technology, orany suitable location-based service or real-time locating system, whichmay use a geo-fence for providing any suitable location-based data toHWD 100 (e.g., to determine a current geo-location of HWD 100 and/or anyother suitable associated data (e.g., the current location is a library,the current location is outside, the current location is your home,etc.)).

Power supply assembly 108 may include any suitable circuitry forreceiving and/or generating power, and for providing such power to oneor more of the other assemblies of HWD 100. For example, power supplyassembly 108 can be coupled to a power grid (e.g., when HWD 100 is notacting as a portable device or when a battery of the device is beingcharged at an electrical outlet with power generated by an electricalpower plant). As another example, power supply assembly 108 may beconfigured to generate power from a natural source (e.g., solar powerusing solar cells). As another example, power supply assembly 108 caninclude one or more batteries for providing power (e.g., when HWD 100 isacting as a portable device).

One or more input assemblies 110 may be provided to permit a user ordevice environment to interact or interface with HWD 100. For example,input assembly 110 can take a variety of forms, including, but notlimited to, a touch pad, dial, click wheel, scroll wheel, touch screen,one or more buttons (e.g., a keyboard), mouse, joy stick, track ball,microphone, camera, scanner (e.g., a barcode scanner or any othersuitable scanner that may obtain product identifying information from acode, such as a linear barcode, a matrix barcode (e.g., a quick response(“QR”) code), or the like), proximity sensor, light detector,temperature sensor, motion sensor, biometric sensor (e.g., a fingerprintreader or other feature (e.g., facial) recognition sensor, which mayoperate in conjunction with a feature-processing application that may beaccessible to HWD 100 for authenticating a user), line-in connector fordata and/or power, and combinations thereof. Each input assembly 110 canbe configured to provide one or more dedicated control functions formaking selections or issuing commands associated with operating HWD 100.Each input assembly 110 may be positioned at any suitable location atleast partially within a space defined by a housing 101 of HWD 100and/or at least partially on an external surface of housing 101 of HWD100.

HWD 100 may also include one or more output assemblies 112 that maypresent information (e.g., graphical, audible, olfactory, and/or tactileinformation) to a user of HWD 100. For example, output assembly 112 ofHWD 100 may take various forms, including, but not limited to, audiospeakers, headphones, line-out connectors for data and/or power, visualdisplays (e.g., for transmitting data via visible light and/or viainvisible light), infrared ports, flashes (e.g., light sources forproviding artificial light for illuminating an environment of thedevice), tactile/haptic outputs (e.g., rumblers, vibrators, etc.), andcombinations thereof. As a specific example, HWD 100 may include adisplay assembly output assembly as output assembly 112, where such adisplay assembly output assembly may include any suitable type ofdisplay or interface for presenting visual data to a user with visiblelight.

It is noted that one or more input assemblies and one or more outputassemblies may sometimes be referred to collectively herein as aninput/output (“I/O”) assembly or I/O interface (e.g., input assembly 110and output assembly 112 as I/O assembly or user interface assembly orI/O interface 111). For example, input assembly 110 and output assembly112 may sometimes be a single I/O interface 111, such as a touch screen,that may receive input information through a user's touch of a displayscreen and that may also provide visual information to a user via thatsame display screen.

Sensor assembly 114 may include any suitable sensor or any suitablecombination of sensors or sensor assemblies that may be operative todetect movements of HWD 100 and/or of a user thereof and/or any othercharacteristics of HWD 100 and/or of its environment (e.g., physicalactivity or other characteristics of a user of HWD 100, light content ofthe device environment, gas pollution content of the device environment,noise pollution content of the device environment, etc.). Sensorassembly 114 may include any suitable sensor(s), including, but notlimited to, one or more of a GPS sensor, accelerometer, directionalsensor (e.g., compass), gyroscope, motion sensor, pedometer, passiveinfrared sensor, ultrasonic sensor, microwave sensor, a tomographicmotion detector, a camera, a biometric sensor, a light sensor, a timer,or the like.

Sensor assembly 114 may include any suitable sensor components orsubassemblies for detecting any suitable movement of HWD 100 and/or of auser thereof. For example, sensor assembly 114 may include one or morethree-axis acceleration motion sensors (e.g., an accelerometer) that maybe operative to detect linear acceleration in three directions (i.e.,the x- or left/right direction, the y- or up/down direction, and the z-or forward/backward direction). As another example, sensor assembly 114may include one or more single-axis or two-axis acceleration motionsensors that may be operative to detect linear acceleration only alongeach of the x- or left/right direction and the y- or up/down direction,or along any other pair of directions. In some embodiments, sensorassembly 114 may include an electrostatic capacitance (e.g.,capacitance-coupling) accelerometer that may be based on siliconmicro-machined micro electro-mechanical systems (“MEMS”) technology,including a heat-based MEMS type accelerometer, a piezoelectric typeaccelerometer, a piezo-resistance type accelerometer, and/or any othersuitable accelerometer (e.g., which may provide a pedometer or othersuitable function). Sensor assembly 114 may be operative to directly orindirectly detect rotation, rotational movement, angular displacement,tilt, position, orientation, motion along a non-linear (e.g., arcuate)path, or any other non-linear motions. Additionally or alternatively,sensor assembly 114 may include one or more angular rate, inertial,and/or gyro-motion sensors or gyroscopes for detecting rotationalmovement. For example, sensor assembly 114 may include one or morerotating or vibrating elements, optical gyroscopes, vibratinggyroscopes, gas rate gyroscopes, ring gyroscopes, magnetometers (e.g.,scalar or vector magnetometers), compasses, and/or the like. Any othersuitable sensors may also or alternatively be provided by sensorassembly 114 for detecting motion on HWD 100, such as any suitablepressure sensors, altimeters, or the like. Using sensor assembly 114,HWD 100 may be configured to determine a velocity, acceleration,orientation, and/or any other suitable motion attribute of HWD 100.

Sensor assembly 114 may include any suitable sensor components orsubassemblies for detecting any suitable biometric data and/or healthdata and/or sleep data and/or mindfulness data and/or the like of a userof HWD 100. For example, sensor assembly 114 may include any suitablebiometric sensor that may include, but is not limited to, one or morehealth-related optical sensors, capacitive sensors, thermal sensors,electric field (“eField”) sensors, and/or ultrasound sensors, such asphotoplethysmogram (“PPG”) sensors, electrocardiography (“ECG”) sensors,galvanic skin response (“GSR”) sensors, posture sensors, stress sensors,photoplethysmogram sensors, and/or the like. These sensors can generatedata providing health-related information associated with the user. Forexample, PPG sensors can provide information regarding a user'srespiratory rate, blood pressure, heart rate (“HR”), heart ratevariability (“HRV”), and/or oxygen saturation. ECG sensors can provideinformation regarding a user's heartbeats. GSR sensors can provideinformation regarding a user's skin moisture, which may be indicative ofsweating and can prioritize a thermostat application to determine auser's body temperature. In some examples, each sensor can be a separatedevice, while, in other examples, any combination of two or more of thesensors can be included within a single device. For example, agyroscope, accelerometer, photoplethysmogram, galvanic skin responsesensor, and temperature sensor can be included within a wearableelectronic device, such as an HWD, while a scale, blood pressure cuff,blood glucose monitor, SpO2 sensor, respiration sensor, posture sensor,stress sensor, and asthma inhaler can each be separate devices. Whilespecific examples are provided, it should be appreciated that othersensors can be used and other combinations of sensors can be combinedinto a single device. Using one or more of these sensors, HWD 100 candetermine physiological characteristics of the user while performing adetected activity, such as a heart rate of a user associated with thedetected activity, average body temperature of a user detected duringthe detected activity, any normal or abnormal physical conditionsassociated with the detected activity, or the like. In some examples, aGPS sensor or any other suitable location detection component(s) of HWD100 can be used to determine a user's location (e.g., geo-locationand/or address and/or location type (e.g., library, school, office, zoo,etc.) and movement, as well as a displacement of the user's motion. Anaccelerometer, directional sensor, and/or gyroscope can further generateactivity data that can be used to determine whether a user of HWD 100 isengaging in an activity, is inactive, or is performing a gesture. Anysuitable activity of a user may be tracked by sensor assembly 114,including, but not limited to, steps taken, flights of stairs climbed,calories burned, distance walked, distance run, minutes of exerciseperformed and exercise quality, time of sleep and sleep quality,nutritional intake (e.g., foods ingested and their nutritional value),mindfulness activities and quantity and quality thereof (e.g., readingefficiency, data retention efficiency), any suitable workaccomplishments of any suitable type (e.g., as may be sensed or loggedby user input information indicative of such accomplishments), and/orthe like. HWD 100 can further include a timer that can be used, forexample, to add time dimensions to various attributes of the detectedphysical activity, such as a duration of a user's physical activity orinactivity, time(s) of a day when the activity is detected or notdetected, and/or the like.

Sensor assembly 114 may include any suitable sensor components orsubassemblies for detecting any suitable characteristics of any suitablecondition of the lighting of the environment of HWD 100. For example,sensor assembly 114 may include any suitable light sensor that mayinclude, but is not limited to, one or more ambient visible light colorsensors, illuminance ambient light level sensors, ultraviolet (“UV”)index and/or UV radiation ambient light sensors, and/or the like. Anysuitable light sensor or combination of light sensors may be providedfor determining the illuminance or light level of ambient light in theenvironment of device 100 (e.g., in lux or lumens per square meter,etc.) and/or for determining the ambient color or white pointchromaticity of ambient light in the environment of device 100 (e.g., inhue and colorfulness or in x/y parameters with respect to an x-ychromaticity space, etc.) and/or for determining the UV index or UVradiation in the environment of device 100 (e.g., in UV index units,etc.). A suitable light sensor may include, for example, a photodiode, aphototransistor, an integrated photodiode and amplifier, or any othersuitable photo-sensitive device. In some embodiments, more than onelight sensor may be integrated into HWD 100. For example, multiplenarrowband light sensors may be integrated into HWD 100 and each lightsensor may be sensitive in a different portion of the light spectrum(e.g., three narrowband light sensors may be integrated into a singlesensor package: a first light sensor may be sensitive to light in thered or infrared region of the electromagnetic spectrum; a second lightsensor may be sensitive in a blue region of the electromagneticspectrum; and a third light sensor may be sensitive in the green portionof the electromagnetic spectrum). Additionally or alternatively, one ormore broadband light sensors may be integrated into HWD 100. The sensingfrequencies of each narrowband sensor may also partially overlap, ornearly overlap, that of another narrowband sensor. Each of the broadbandlight sensors may be sensitive to light throughout the spectrum ofvisible light and the various ranges of visible light (e.g., red, green,and blue ranges) may be filtered out so that a determination may be madeas to the color of the ambient light. As used herein, “white point” mayrefer to coordinates in a chromaticity curve that may define the color“white.” For example, a plot of a chromaticity curve from the CommissionInternational de l'Eclairage (“CIE”) may be accessible to system 1(e.g., as a portion of data stored by memory assembly 104), wherein thecircumference of the chromaticity curve may represent a range ofwavelengths in nanometers of visible light and, hence, may representtrue colors, whereas points contained within the area defined by thechromaticity curve may represent a mixture of colors. A Planckian curvemay be defined within the area defined by the chromaticity curve and maycorrespond to colors of a black body when heated. The Planckian curvepasses through a white region (i.e., the region that includes acombination of all the colors) and, as such, the term “white point” issometimes generalized as a point along the Planckian curve resulting ineither a bluish white point or a yellowish white point. However, “whitepoint” may also include points that are not on the Planckian curve. Forexample, in some cases the white point may have a reddish hue, agreenish hue, or a hue resulting from any combination of colors. Theperceived white point of light sources may vary depending on the ambientlighting conditions in which the lights source is operating. Such achromaticity curve plot may be used in coordination with any sensedlight characteristics to determine the ambient color (e.g., true color)and/or white point chromaticity of the environment of HWD 100 in anysuitable manner. Any suitable UV index sensors and/or ambient colorsensors and/or illuminance sensors may be provided by sensor assembly114 in order to determine the current UV index and/or chromaticityand/or illuminance of the ambient environment of device 100.

Sensor assembly 114 may include any suitable sensor components orsubassemblies for detecting any suitable characteristics of any suitablecondition of the air quality of the environment of HWD 100. For example,sensor assembly 114 may include any suitable air quality sensor that mayinclude, but is not limited to, one or more ambient air flow or airvelocity meters, ambient oxygen level sensors, volatile organic compound(“VOC”) sensors, ambient humidity sensors, ambient temperature sensors,and/or the like. Any suitable ambient air sensor or combination ofambient air sensors may be provided for determining the oxygen level ofthe ambient air in the environment of HWD 100 (e.g., in O₂% per liter,etc.) and/or for determining the air velocity of the ambient air in theenvironment of HWD 100 (e.g., in kilograms per second, etc.) and/or fordetermining the level of any suitable harmful gas or potentially harmfulsubstance (e.g., VOC (e.g., any suitable harmful gasses, scents, odors,etc.) or particulate or dust or pollen or mold or the like) of theambient air in the environment of HWD 100 (e.g., in HG % per liter,etc.) and/or for determining the humidity of the ambient air in theenvironment of HWD 100 (e.g., in grams of water per cubic meter, etc.(e.g., using a hygrometer)) and/or for determining the temperature ofthe ambient air in the environment of HWD 100 (e.g., in degrees Celsius,etc. (e.g., using a thermometer)).

Sensor assembly 114 may include any suitable sensor components orsubassemblies for detecting any suitable characteristics of any suitablecondition of the sound quality of the environment of HWD 100. Forexample, sensor assembly 114 may include any suitable sound qualitysensor that may include, but is not limited to, one or more microphonesor the like that may determine the level of sound pollution or noise inthe environment of HWD 100 (e.g., in decibels, etc.). Sensor assembly114 may also include any other suitable sensor for determining any othersuitable characteristics about a user of HWD 100 and/or otherwise aboutthe environment of device 100 and/or any situation within which HWD 100may be existing. For example, any suitable clock and/or positionsensor(s) may be provided to determine the current time and/or time zonewithin which HWD 100 may be located.

One or more sensors or sensor subassemblies of sensor assembly 114 maybe embedded in a structural body (e.g., housing 101) of HWD 100, such asalong a bottom surface that may be operative to contact a user, or canbe positioned at any other desirable location. In some examples,different sensors can be placed in different locations inside or on thesurfaces of HWD 100 (e.g., some located inside housing 101 and someattached to an attachment mechanism (e.g., a wrist band coupled to ahousing of a wearable device), or the like). In other examples, one ormore sensors can be worn by a user separately as different parts of asingle HWD 100 or as different HWDs (e.g., as a pair of earrings). Insuch cases, the sensors can be configured to communicate with HWD 100using a wired and/or wireless technology (e.g., via communicationsassembly 106). In some examples, sensors can be configured tocommunicate with each other and/or share data collected from one or moresensors. In some examples, HWD 100 can be waterproof such that thesensors can detect a user's activity in water.

Processor assembly 102 of HWD 100 may include any processing circuitrythat may be operative to control the operations and performance of oneor more assemblies of HWD 100. For example, processor assembly 102 mayreceive input signals from input assembly 110 and/or drive outputsignals through output assembly 112. As shown in FIG. 1, processorassembly 102 may be used to run one or more applications, such as anapplication 103. Application 103 may include, but is not limited to, oneor more operating system applications, firmware applications, mediaplayback applications, media editing applications, pass applications,calendar applications, state determination applications, biometricfeature-processing applications, compass applications, healthapplications, mindfulness applications, sleep applications, thermometerapplications, weather applications, thermal management applications,video game applications, comfort applications, device and/or useractivity applications, or any other suitable applications. For example,processor assembly 102 may load application 103 as a user interfaceprogram to determine how instructions or data received via an inputassembly 110 and/or sensor assembly 114 and/or any other assembly of HWD100 (e.g., any suitable auxiliary subsystem data 91 that may be receivedby HWD 100 via communications assembly 106) may manipulate the one ormore ways in which information may be stored on HWD 100 and/or providedto a user via an output assembly 112 and/or provided to an auxiliarysubsystem (e.g., to subsystem 200 as auxiliary subsystem data 99 viacommunications assembly 106). Application 103 may be accessed byprocessor assembly 102 from any suitable source, such as from memoryassembly 104 (e.g., via bus 116) or from another remote device or server(e.g., from subsystem 200 of system 1 via communications assembly 106).Processor assembly 102 may include a single processor or multipleprocessors. For example, processor assembly 102 may include at least one“general purpose” microprocessor, a combination of general and specialpurpose microprocessors, instruction set processors, graphicsprocessors, video processors, and/or related chips sets, and/or specialpurpose microprocessors. Processor assembly 102 also may include onboard memory for caching purposes.

One particular type of application available to processor assembly 102may be an activity application 103 that may be operative to determine orpredict a current or planned activity of device 100 and/or for a userthereof. Such an activity may be determined by activity application 103based on any suitable data accessible by activity application 103 (e.g.,from memory assembly 104 and/or from any suitable remote entity (e.g.,any suitable auxiliary subsystem data 91 from any suitable auxiliarysubsystem 200 via communications assembly 106)), such as data from anysuitable activity data source, including, but not limited to, a calendarapplication, a gaming application, a media playback application, ahealth application, a social media application, an exercise monitoringapplication, a sleep monitoring application, a mindfulness monitoringapplication, transaction information, wireless connection information,subscription information, contact information, pass information, and/orthe like. For example, at a particular time, such an activityapplication 103 may be operative to determine one or more currentactivities of a user wearing HWD 100, such as exercise, sleep, eat,study, read, relax, play, and/or the like. Alternatively, such anactivity application 103 may request that a user indicate a currentactivity (e.g., via a user interface assembly).

HWD 100 may also be provided with housing 101 that may at leastpartially enclose at least a portion of one or more of the assemblies ofHWD 100 for protection from debris and other degrading forces externalto HWD 100. In some embodiments, one or more of the assemblies may beprovided within its own housing (e.g., a first sensor assembly may bepositioned within a frame structure housing that may be holding aneyeglass, while a second sensor assembly may be positioned in or about astrap that may be coupled to the frame structure and extend about theback of the user's head, and/or a first sensor assembly may bepositioned within a first earring or earbud structure housing that maybe worn by a first ear of a user, while a second sensor assembly may bepositioned within a second earring or earbud structure housing that maybe worn by a second ear of the user, and/or an input assembly 110 may bean independent keyboard or mouse within its own housing that maywirelessly or through a wire communicate with processor assembly 102,which may be provided within its own housing).

Processor assembly 102 may load any suitable application 103 as abackground application program or a user-detectable application programin conjunction with any suitable head gesture cluster data 105 or anyother suitable data (e.g., data 91 from subsystem 200) to determine howany suitable input assembly data received via any suitable inputassembly 110 and/or any suitable sensor assembly data received via anysuitable sensor assembly 114 and/or any other suitable data received viaany other suitable assembly of device 100 (e.g., any suitable auxiliarysubsystem data 91 received from auxiliary subsystem 200 viacommunications assembly 106 of HWD 100) may be used to determine anysuitable user state data (e.g., user state data 322 of FIG. 3) that maybe used to control or manipulate at least one functionality of HWD 100(e.g., a performance or mode of HWD 100 that may be altered in aparticular one of various ways (e.g., particular user alerts orrecommendations may be provided to a user via a user interface assemblyand/or particular adjustments may be made by an output assembly and/orthe like)) and/or at least one functionality of subsystem 200 orotherwise of system 1.

System 1 may include one or more auxiliary electronic subsystems 200that may include any suitable assemblies, such as assemblies that may besimilar to one, some, or each of the assemblies of HWD 100. As shown inFIG. 1, for example, auxiliary electronic subsystem 200 may include aprocessor assembly 202, an application 203, a memory assembly 204, data205, a communications assembly 206, a power supply assembly 208, aninput assembly 210, an I/O assembly 211, an output assembly 212, asensor assembly 214, and a bus 216. In some embodiments, one or moreassemblies of auxiliary electronic subsystem 200 may be combined oromitted. Moreover, auxiliary electronic subsystem 200 may include anyother suitable assemblies not combined or included in FIG. 1 and/orseveral instances of the assemblies shown in FIG. 1. For the sake ofsimplicity, only one of each of the assemblies is shown in FIG. 1.Subsystem 200 may be configured to work in conjunction with or otherwiseto be paired with or be a companion to HWD 100 in any suitable manner(e.g., to share processing capabilities and/or memory storagecapabilities). Subsystem 200 may be configured to communicate anysuitable auxiliary subsystem data 91 to HWD 100 (e.g., viacommunications assembly 206 of subsystem 200 and communications assembly106 of HWD 100), such as automatically and/or in response to anauxiliary subsystem data request of data 99 that may be communicatedfrom HWD 100 to auxiliary subsystem 200.

Auxiliary electronic subsystem 200 can include, but is not limited to, amusic player (e.g., an iPod™ available by Apple Inc. of Cupertino,Calif.), video player, still image player, game player, other mediaplayer, music recorder, movie or video camera or recorder, still camera,other media recorder, radio, medical equipment, domestic appliance,transportation vehicle instrument, musical instrument, calculator,cellular telephone (e.g., an iPhone™ available by Apple Inc.), otherwireless communication device, wearable device (e.g., an Apple Watch™available by Apple Inc.), personal digital assistant, remote control,pager, computer (e.g., a desktop (e.g., an iMac™ available by AppleInc.), laptop (e.g., a MacBook™ available by Apple Inc.), tablet (e.g.,an iPad™ available by Apple Inc.), server, etc.), monitor, television,stereo equipment, set up box, set-top box, gaming console, boom box,modem, router, printer, controller (e.g., game controller), or anycombination thereof. In some embodiments, auxiliary electronic subsystem200 may perform a single function (e.g., a subsystem dedicated toprocessing certain data from and/or for HWD 100) and, in otherembodiments, auxiliary electronic subsystem 200 may perform multiplefunctions (e.g., a subsystem that processes data from and/or for HWD100, plays music, and receives and transmits telephone calls). Auxiliaryelectronic subsystem 200 may be any portable, mobile, hand-held, orminiature electronic device that may be configured to function incooperating with HWD 100 wherever a user travels. Some miniatureelectronic devices may have a form factor that is smaller than that ofhand-held electronic devices, such as an iPod™. Illustrative miniatureelectronic devices can be integrated into various objects that mayinclude, but are not limited to, watches (e.g., an Apple Watch™available by Apple Inc.), rings, necklaces, belts, accessories forbelts, headsets, accessories for shoes, virtual reality devices,glasses, other wearable electronics, accessories for sporting equipment,accessories for fitness equipment, key chains, or any combinationthereof. Alternatively, auxiliary electronic subsystem 200 may not beportable at all, but may instead be generally stationary.

FIGS. 2-2H show system 1, where, as just one specific example, HWD 100may be provided as any suitable set of eyeglasses worn by a user U on ahead H with right ear ER, left ear EL, right eye YR, left eye YL, noseN, and mouth M, while auxiliary electronic subsystem 200 may be ahand-held or otherwise portable electronic device, such as an iPhone™,that may be carried by or otherwise brought with user U wherever ittravels (FIG. 2I may show just one other illustrative system 1′, wherean HWD 100′ may be provided as a virtual reality head-mounted displaydevice with one or more sensor assemblies 114′ for positioning a displayover the eyes of user U and holding HWD 100′ to the user's head with oneor more straps 101 sp′, and where subsystem 200′ may be any suitablegaming controller). As shown, eyeglasses HWD 100 of FIGS. 2-2H mayinclude an eyeglass housing structure 101 hs with an interior structuresurface 101 ih and an exterior structure surface 101 eh, where thestructure of eyeglass housing 101 hs may include a right eye frame 101fr supporting a right eye glass lens 101 gr, a left eye frame 101 flsupporting a left eye glass lens 101 gl, a bridge 101 b connecting eyeframes 101 fr and 101 fl, a right hinge 101 hr coupling right eye frame101 fr to a right temple frame 101 tr that extends from right hinge 101hr to a right temple tip 101 pr, and a left hinge 101 hl coupling lefteye frame 101 fl to a left temple frame 10 tl that extends from lefthinge 101 hl to a left temple tip 101 pl, such that, when worn on head Hof user U, bridge 101 b may rest on a portion of nose N, right templeframe 101 tr may rest on a portion of right ear ER, and left templeframe 101 tl may rest on a portion of left ear EL, whereby right eyeglass lens 101 gr may be aligned with right eye YR and left eye glasslens 101 gl may be aligned with left eye YL. The structure of eyeglasshousing 101 hs may be configured with any suitable shape and geometryand bias for being comfortably and/or securely worn by user U's head H,such that at least a portion of housing 101 hs and/or any other portionof HWD 100 may be held at least partially against skin HS (e.g.,epidermis, dermis, hypodermis, and/or subcutaneous tissue with orwithout hair) of head H, which surrounds skull HK protecting brain HBand/or any other suitable anatomy. For example, temple frames 101 tr and101 tl of eyeglass housing 101 hs may be configured as spring-loadedflex hinge temples that may be coupled to flex hinges equipped with asmall spring that may afford the temple arms a greater range of movementand that may not limit them to a traditional (e.g., 90 degree) angle, asskull temples that may be operative to bend down behind the ears andfollow the contour of the skull and rest evenly against the skull'sskin, as library temples that may be generally straight and do not benddown behind the ears but that may hold the glasses primarily throughlight pressure against the side of the skull's skin, as convertibletemples that may be used either as library or skull temples depending onthe bent structure, as riding bow temples that may curve around the earand extend down to the level of the ear lobe (e.g., as may be commonlyused on athletic, children's, and industrial safety frames), as comfortcable temples that may be similar to a riding bow but constructed fromcoiled, metal, flexible cable, and/or as any other suitable temples. Anysuitable materials, such as any suitable plastic, metal, wood, bone,ivory, leather, stone, or any combination thereof, may be used toprovide various portions of eyeglass housing 101 hs. As shown in FIG.2B, for example, a strap 101 sp (e.g., an elastic strap (e.g., a stretchfabric)) or any other suitable mechanism may be provided to extend aboutthe back side of head H, such as between tips 101 pr and 101 pl, forfurther securing HWD 100 against head H during use. As also shown, atleast one ambient light source AS may exist in the environment of HWD100 that may be emitting ambient light AL (e.g., towards the right sideof head H with right ear ER).

HWD 100 may include one or more light-sensing assemblies, such aslight-sensing assemblies 114 a-114 i, positioned along eyeglass housing101 hs, each of which may be operative to detect light reflected byand/or transmitted through a portion of the head H (e.g., a portion ofskin user's head skin HS, a portion of right ear ER, a portion of leftear EL, etc.). For example, as shown, light-sensing assemblies 114 a-114d may be provided at different locations along the length of righttemple frame 101 tr between right temple tip 101 pr and right hinge 101hr for potentially interfacing with different respective skin portionsHSa-HSd along the right side of the face of user U and/or with right earER, light-sensing assembly 114 e may be provided at bridge 101 b forpotentially interfacing with respective skin portion HSe along theforehead or bridge of the nose of the face of user U and/or with nose N,and light-sensing assemblies 114 f-114 i may be provided at differentlocations along the length of left temple frame 101 tl between lefthinge 101 hl and left temple tip 101 pl for potentially interfacing withdifferent respective skin portions HSf-HSi along the left side of theface of user U and/or with left ear EL (and, although not shown, one ormore light-sensing assemblies may be positioned at any other suitablelocation along HWD 100, including along any suitable portion(s) of strap101 sp for potentially interfacing with any suitable portion(s) of auser's neck or rear skull or the like). Each light-sensing assembly mayinclude at least one light-sensing component (e.g., a photodiode) andeach light-sensing assembly may be provided about a portion of eyeglasshousing 101 hs, within a portion of eyeglass housing 101 hs, and/orthrough a portion of eyeglass housing 101 hs such that, when eyeglasshousing 101 hs is worn on user's head H, at least one, some, or eachlight-sensing component of at least one, some, or each light-sensingassembly may be operative to face, contact, or otherwise be positionedrelative to a respective tissue or skin portion of the user's head(e.g., skin portions HSa-HSi) in order to detect light that may bereflected by and/or transmitted through the skin portion, and such thatHWD 100 (e.g., processor assembly 102) may utilize the sensor datadetected by the light sensing component(s) to determine any suitablehead gestures of user U and/or any suitable biological and/orphysiological characteristics (e.g., heart rate characteristics) of userU. Therefore, each light-sensing assembly 114 may be operative to detectlight sensor data that may vary according to the periodic motion ofblood through human head tissue, which may be used to detect avolumetric measurement of a blood vessel or any suitable opticallyobtained plethysmogram for use in photoplethysmography (“PPG”) that maybe used to detect any suitable heart rate or other physiological data ofthe user and/or to determine a head gesture of the user. Detected light° sensor data may be sensitive to blood volume variations (e.g., bloodflow variations) at the portion of the user's head that may bereflecting the detected light that may be at least partially definingthe light sensor data.

As shown in more detail in FIG. 2C, light-sensing assembly 114 b mayinclude any number of various sensor components, such as a firstlight-sensing component 124 (e.g., a first photodiode (“PD1”)), a secondlight-sensing component 134 (e.g., a second photodiode (“PD2”)), a thirdlight-sensing component 144 (e.g., a third photodiode (“PD3”)), one,some, or each of which may be positioned within a portion of eyeglasshousing 101 hs (e.g., within a portion of right temple frame 101 tr) butmay be exposed to a portion of skin HS (e.g., a portion of skin surfaceHSs along skin portion HSb), such as directly or via alight-transmissive opening or element. For example, first light-sensingcomponent 124 may be exposed to and operative to detect a first devicelight DL1 that may be emitted from skin surface HSs of skin portion HSbof skin HS via a first light-transmissive element 125 that may extend atleast between first light-sensing component 124 and interior structuresurface 101 ih of eyeglass housing 101 hs, second light-sensingcomponent 134 may be exposed to and operative to detect a second devicelight DL2 that may be emitted from skin surface HSs of skin portion HSbof skin HS via a second light-transmissive element 135 that may extendat least between second light-sensing component 134 and interiorstructure surface 101 ih of eyeglass housing 101 hs, and/or thirdlight-sensing component 144 may be exposed to and operative to detect athird device light DL3 that may be emitted from skin surface HSs of skinportion HSb of skin HS via a third light-transmissive element 145 thatmay extend at least between third light-sensing component 144 andinterior structure surface 101 ih of eyeglass housing 101 hs.Light-sensing assembly 114 b may also include any number of variouslight-emitting components, including a first light-emitting component154 (e.g., a first light emitting diode (“LE1”)), which may bepositioned within a portion of eyeglass housing 101 hs (e.g., within aportion of right temple frame 101 tr) but exposed to a portion of skinHS (e.g., a portion of skin surface HSs of skin portion HSb) directly orvia a light-transmissive opening or element. For example, light-emittingcomponent 154 may be operative to transmit each one of first devicelight DL1, second device light DL2, and third device light DL3 from HWD100 and into skin HS through skin surface HSs of skin portion HSb viaone or more light-transmissive elements 155 (e.g., a singlelight-transmissive element or respective different light-transmissiveelements for the different emitted lights) that may extend at leastbetween light-emitting component 154 and interior structure surface 101ih of eyeglass housing 101 hs. Additionally or alternatively,light-sensing assembly 114 b may include any number of various othersensor components, such as first additional-sensing component 164 (e.g.,“AS1”, which may be a sound sensor (e.g., piezo or other microphone) orany other suitable sensor) and a second additional-sensing component 174(e.g., “AS2”, which may be a force or contact or pressure sensor or anyother suitable sensor), each of which may be provided along withcomponents 124, 134, 144, and 154 along a particular portion of eyeglasshousing 101 hs. As also shown, in some embodiments, light-sensingassembly 114 b may also include any suitable movement output assembly122 (e.g., a “MOTOR”, such as any suitable piezo motor) that may beoperative to adjust the position of light-sensing assembly 114 b alongeyeglass housing 101 hs (e.g., movement output assembly 122 may beoperative to move at least a portion or the entirety of light-sensingassembly 114 b in the direction of arrow 122 u along right temple frame101 tr towards right temple tip 101 pr and/or in the direction of arrow122 d along right temple frame 101 tr towards right hinge 101 hr and/orin the direction of arrow 122 i towards skin HS and/or in the directionof arrow 122 e towards right ear ER and/or in any other direction orrotation (e.g., about an axis of right temple frame 101 tr (e.g., aboutan axis of arrow 122 u) or about an axis of arrow 122 e or about oralong an axis perpendicular to each one of arrows 122 u and 122 e). Anysuitable geometry may be used between components of a light-sensingassembly. For example, a length of an array of sensor components of alight-sensing assembly (e.g., a distance between AS1 and PD3 of assembly114 b) may be any suitable distance, such as between 10 millimeters and50 millimeters, while a spacing between any two components (e.g., adistance between PD1 and LE1 of assembly 114 b) may be any suitabledistance, such as between 1 millimeter and 7 millimeters.

As also shown in more detail in FIG. 2C, because light-sensing assembly114 b may not only interface with skin portion HSb of skin HS but alsoskin portion ERSb of right ear ER, light-sensing assembly 114 b mayinclude any number of various other sensor components, including a firstother light-sensing component 124′ (e.g., a first other photodiode(“PD1′”)), a second other light-sensing component 134′ (e.g., a secondother photodiode (“PD2′”)), a third other light-sensing component 144′(e.g., a third other photodiode (“PD3′”)), each of which may bepositioned within a portion of eyeglass housing 101 hs (e.g., within aportion of right temple frame 101 tr) but exposed to a portion of skinportion ERSb of right ear ER directly or via a light-transmissiveopening or element. For example, first other light-sensing component124′ may be exposed to and operative to detect each one of a first otherdevice light DL1′ and a first ambient light AL1 that may be emitted fromskin surface ERis of skin portion ERSb of right ear ER via a first otherlight-transmissive element 125′ that may extend at least between firstother light-sensing component 124′ and exterior structure surface 101 ehof eyeglass housing 101 hs, second other light-sensing component 134′may be exposed to and operative to detect each one of a second otherdevice light DL2′ and a second ambient light AL2 that may be emittedfrom skin surface ERis of skin portion ERSb of right ear ER via a secondother light-transmissive element 135′ that may extend at least betweensecond other light-sensing component 134′ and exterior structure surface101 eh of eyeglass housing 101 hs, and/or third other light-sensingcomponent 144′ may be exposed to and operative to detect each one of athird other device light DL3′ and a third ambient light AL3 that may beemitted from skin surface ERis of skin portion ERSb of right ear ER viaa third other light-transmissive element 145′ that may extend at leastbetween third other light-sensing component 144′ and exterior structuresurface 101 eh of eyeglass housing 101 hs. Light-sensing assembly 114 bmay also include any number of other various light-emitting components,including a first other light-emitting component 154′ (e.g., a firstlight emitting diode (“LE1′”)), which may be positioned within a portionof eyeglass housing 101 hs (e.g., within a portion of right temple frame101 tr) but exposed to a portion of skin surface ERis of skin portionERSb of right ear ER directly or via another light-transmissive openingor element. For example, other light-emitting component 154′ may beoperative to transmit each one of first other device light DL1′, secondother device light DL2′, and third other device light DL3′ from HWD 100and into skin portion ERSb of right ear ER through skin surface ERis viaone or more other light-transmissive elements 155′ (e.g., a singlelight-transmissive element or respective different light-transmissiveelements for the different emitted lights) that may extend at leastbetween other light-emitting component 154′ and exterior structuresurface 101 eh of eyeglass housing 101 hs. Additionally oralternatively, light-sensing assembly 114 b may include any number ofvarious other sensor components, such as first other additional-sensingcomponent 164′ (e.g., “AS1′”, which may be a sound sensor (e.g., piezoor other microphone) or any other suitable sensor) and a second otheradditional-sensing component 174′ (e.g., “AS2′”, which may be a force orcontact or pressure sensor or any other suitable sensor), each of whichmay be provided along with components 124′, 134′, 144′, and 154′ along aparticular portion of eyeglass housing 101 hs.

Each light-sensing assembly may be provided about a portion of eyeglasshousing 101 hs, within a portion of eyeglass housing 101 hs, and/orthrough a portion of eyeglass housing 101 hs such that, when eyeglasshousing 101 hs is worn on user's head H, at least one, some, or eachlight-sensing component of at least one, some, or each light-sensingassembly may be operative to face, contact, or otherwise be positionedrelative to a respective tissue or skin portion of the user's head inorder to detect light that may be reflected by and/or transmittedthrough the skin portion. As shown in FIG. 2C, for example,light-sensing assembly 114 b may be positioned within a portion of righttemple frame 101 tr of eyeglass housing 101 hs and between exteriorstructure surface 101 eh of eyeglass housing 101 hs and interiorstructure surface 101 ih of eyeglass housing 101 hs. As shown in FIG.2D, for example, light-sensing assembly 114 d may be provided in a formfactor that may be provided about a portion of eyeglass housing 101 hs,such as a clip-on form factor with a structure that may include a hinge114 dp about which other portions of the structure about an opening 114do may rotate or otherwise deflect (e.g., in the direction of arrow OP)for adjusting the size of opening 114 do for enabling a portion ofeyeglass housing 101 hs (e.g., a portion of right temple frame 101 tr)to be positioned within a space defined by the structure oflight-sensing assembly 114 d, such that assembly 114 d may be removablycoupled (e.g., clipped onto) and/or slid along a portion of eyeglasshousing 101 hs. Such a dynamic form factor may enable light-sensingassembly 114 d to be placed in an optimized (e.g., comfortable and/oreffective) location. Moreover, as shown, light-sensing assembly 114 d orany other suitable light-sensing assembly may include any suitabledeformable mechanism 114 db (e.g., a spring and/or a foam element and/orany other suitable biasing component) that may be operative to deform(e.g., contract in the direction of arrows CNT) to reduce a dimensionDST between a portion of eyeglass housing 101 hs and an external surface114 di of assembly 114 d and/or to deform (e.g., expand in the directionof arrows EXP) to increase a dimension DST between a portion of eyeglasshousing 101 hs and external surface 114 di of assembly 114 d, which mayenable external surface 114 di to be biased against a surface of headskin HS. As shown in FIG. 2E, for example, light-sensing assembly 114 cmay be positioned with an external surface 114 ci flush with interiorstructure surface 101 ih of eyeglass housing 101 hs. As shown in FIG.2F, for example, light-sensing assembly 114 f may be positioned with anexternal surface 114 fi that is spaced outwardly from interior structuresurface 101 ih of eyeglass housing 101 hs, which may enable improvedcontact with head skin HS and/or increased localized pressure on theuser. As shown in FIG. 2G, for example, light-sensing assembly 114 g maybe positioned at least partially within a foam or otherwise suitablycompliant component 115 g that may extend outwardly at an angle awayfrom interior structure surface 101 ih of eyeglass housing 101 hs, whichmay enable improved contact with head skin HS and/or increased localizedpressure on the user while maintaining user comfort. As shown in FIG.2H, for example, light-sensing assembly 114 h may be positioned at leastpartially within and/or include a foam or otherwise suitably compliantcomponent 115 h that may extend outwardly away from interior structuresurface 101 ih of eyeglass housing 101 hs, which may enable improvedcontact with head skin HS and/or increased localized pressure on theuser while maintaining user comfort. Therefore, various structures,relationships, and/or materials may be used to position a light-sensingassembly with respect to a housing of an HWD so that pressure may beapplied against head skin HS for facilitating better light-sensingtherefrom without compromising user comfort and/or so that a contactpoint of the HWD to a user's head skin is close to or includes alight-sensing assembly with one or more light-sensing components.

Different light-sensing components of a light-sensing assembly 114 ofHWD 100 (e.g., different ones of light-sensing components 124, 124′,134, 134′, 144, and 144′ of light-sensing assembly 114 b) may beconfigured to sense light at a respective different position on a skinsurface of head H of user U when HWD 100 may be worn on head H. Due tothis positioning, the sensor data from each light-sensing component maybe operative to capture movement of anatomical features in the tissue ofthe head skin of the user during any suitable head gesture. In someembodiments, a single light-emitting component or differentlight-emitting components of a light-sensing assembly 114 of HWD 100(e.g., each one of light-emitting components 154 and 154′ oflight-sensing assembly 114 b) may be configured to emit light at one ormore different wavelengths, which may penetrate to different depths inthe head tissue of the user before reflecting back to one or morelight-sensing components of HWD 100 (e.g., light-emitting component 154may be operative to emit first light DL1 as infrared or red light via afirst emitter to at least PD1, to emit second light DL2 as green lightvia a second emitter to at least PD2, and to emit third light DL3 asblue light via a third emitter to at least PD3). In some examples, eachpossible photodiode-emitter combination can be considered a separatechannel of light sensor data. For example, in a device with three lightemitters and three photodiodes (e.g., three light emitters oflight-emitting component 154 and three light-sensing components 124,134, and 144 of light-sensing assembly 114 b with respect to skinportion HSb of skin HS of head H), there can be nine channels of lightsensor data. When a first light emitter of light-emitting component 154emits light, the first, second, and third light-sensing components 124,134, and 144 may sense first, second, and third channels of light sensordata, respectively. When a second light emitter of light-emittingcomponent 154 emits light, the first, second, and third light-sensingcomponents 124, 134, and 144 may sense fourth, fifth, and sixth channelsof light sensor data, respectively. When a third light emitter oflight-emitting component 154 emits light, the first, second, and thirdlight-sensing components 124, 134, and 144 may sense seventh, eighth,and ninth channels of light sensor data, respectively. Eachlight-sensing assembly may also include additional sensor types that maybe operative to provide additional channels of sensor data. For example,first additional-sensing component 164 of light-sensing assembly 114 bmay be a force sensor that may be operative to detect force (e.g., ofthe head) against HWD 100 and may provide a first additional channel ofsensor data, while second additional sensing-component 174 oflight-sensing assembly 114 b may be an accelerometer that may beoperative to sense acceleration of HWD 100 in each one of X-, Y-, andZ-directions and may provide second, third, and fourth additionalchannels of sensor data, respectively. In some examples, additionalchannels of sensor data can include data from a barometer, amagnetometer, a GPS receiver, a microphone, and/or numerous otherpossibilities of sensors. In some examples, other light sensors may beused in place of or in addition to photodiodes. In some examples, aforce sensor can be spatially discretized, sensing force independentlyat multiple positions of the surface of the device that may contact orotherwise interface with a head of a user, in which case the forcesensor can provide multiple (e.g., 4) channels of pressure information.

FIGS. 2 and 4A-4E illustrate just some exemplary head gestures that maybe detected by one or more light-sensing assemblies 114 of HWD 100 inaccordance with examples of the disclosure. In some examples, head U canbe expressionless (e.g., stationary in a resting position), asillustrated in FIG. 2. As just one other example, different portions ofa jawbone JB may be moved in the respective direction of arrows A1, A2,and A3 with respect to a right cheek bone CR and a left cheek bone CLfor providing a leftward lateral jaw excursion, as illustrated in FIG.4A. As just one other example, different portions of jawbone JB may bemoved in the respective direction of arrows A4 and A5 with respect tocheek bone CR for providing a jaw depression, as illustrated in FIG. 4B.As just one other example, different portions of jawbone JB may be movedin the respective direction of arrows A6 and A7 with respect to cheekbone CR for providing a jaw elevation, as illustrated in FIG. 4C. Asjust one other example, different portions of jawbone JB may be moved inthe respective direction of arrows A8 and A9 with respect to cheek boneCR for providing a jaw protrusion, as illustrated in FIG. 4D. As justone other example, different portions of jawbone JB may be moved in therespective direction of arrows A10 and A11 with respect to cheek bone CRfor providing a jaw retrusion, as illustrated in FIG. 4E. The examplehead gestures of FIGS. 2 and 4A-4E are just exemplary and by no meansexhaustive. Various other head gestures, including, but not limited to,chewing, blinking, winking, smiling, eyebrow raising, eyes widening oreyes rolling or eyes squinting or the like, humming or other internalvocalizations (e.g., “mmm-hmm”, “uh-huh”, etc.), inaudible cues, jawmotions, flaring nostrils, speaking or other external explicit languagevocalization, mouth opening (e.g., full mouth opening, left-side mouthopening, right-side mouth opening, etc.), ear wiggling or other earmovement, smirking, frowning, grimacing, cheek motioning, emotionsand/or thoughts and/or brain functions and/or heart rate characteristicsand/or respiratory rate and/or blood pressure and/or heart rate (“HR”)and/or heart rate variability (“HRV”) and/or oxygen saturation and/orother biometric characteristics and/or any other voluntary gesturesand/or any other involuntary gestures (e.g., reactions and/or reactivegestures) or countenance of the user and/or the like may be detected byone or more light-sensing assemblies 114 of HWD 100. Variations of thesegestures and other gestures entirely may be trained on and detected inaccordance with examples of the disclosure.

FIG. 5 illustrates collection 500 of exemplary charts of sensor data501-506 in accordance with examples of the disclosure. For example,charted sensor data 501 may represent a particular channel of lightsensor data of light-sensing assembly 114 b (e.g., the light sensorchannel between a particular emitter of light-emitting component 154 andlight-sensing component 124 of light-sensing assembly 114 b for firstdevice light DL1 with respect to skin portion HSb of skin HS of theright side of head H) during a chewing head gesture (e.g., a period oftime during which the user may repeatedly move between the states ofFIG. 4B and FIG. 4C), while charted sensor data 502 may represent aparticular channel of light sensor data of light-sensing assembly 114 h(e.g., the light sensor channel between a particular emitter of alight-emitting component and a light-sensing component of light-sensingassembly 114 h for a device light with respect to skin portion HSh ofskin HS of the left side of head H) during a chewing head gesture (e.g.,a period of time during which the user may repeatedly move between thestates of FIG. 4B and FIG. 4C). As another example, charted sensor data503 may represent a particular channel of light sensor data oflight-sensing assembly 114 b (e.g., the light sensor channel between aparticular emitter of light-emitting component 154 and light-sensingcomponent 124 of light-sensing assembly 114 b for first device light DL1with respect to skin portion HSb of skin HS of the right side of head H)during an opening and closing of the right side of mouth M head gesture(e.g., a period of time during which the user may repeatedly open andclose only the right side of mouth M), while charted sensor data 504 mayrepresent a particular channel of light sensor data of light-sensingassembly 114 h (e.g., the light sensor channel between a particularemitter of a light-emitting component and a light-sensing component oflight-sensing assembly 114 h for a device light with respect to skinportion HSh of skin HS of the left side of head H) during an opening andclosing of the right side of mouth M head gesture (e.g., a period oftime during which the user may repeatedly open and close only the rightside of mouth M). As another example, charted sensor data 505 mayrepresent a particular channel of light sensor data of light-sensingassembly 114 b (e.g., the light sensor channel between a particularemitter of light-emitting component 154 and light-sensing component 124of light-sensing assembly 114 b for first device light DL1 with respectto skin portion HSb of skin HS of the right side of head H) during astationary head gesture (e.g., a period of time during which the usermay remain in the state of FIG. 2), while charted sensor data 506 mayrepresent a particular channel of light sensor data of light-sensingassembly 114 h (e.g., the light sensor channel between a particularemitter of a light-emitting component and a light-sensing component oflight-sensing assembly 114 h for a device light with respect to skinportion HSh of skin HS of the left side of head H) during a stationaryhead gesture (e.g., a period of time during which the user may remain inthe state of FIG. 2). As just one example, each light sensor datachannel may be provided by an IR light emitter and each charted sensordata may be band pass filtered. As can be observed, during the chewinghead gesture, charted sensor data 501 of the light sensor data channelof assembly 114 b of the right side of head H and charted sensor data502 of the light sensor data channel of assembly 114 h of the left sideof head H may exhibit somewhat similar signal characteristics (e.g., asthe left side channel and the right side channel may be similarlyaffected by a chewing head gesture). Additionally, as can be observed,during the opening and closing of the right side of mouth M headgesture, charted sensor data 503 of the light sensor data channel ofassembly 114 b of the right side of head H may exhibit significantlydifferent signal characteristics (e.g., significantly larger amplitudedifference between an adjacent peak and trough) than the signalcharacteristics exhibited by charted sensor data 504 of the light sensordata channel of assembly 114 h of the left side of head H (e.g., as theright side of the head and, thus, the right side channel, may be moresignificantly affected by an opening and closing of the right side ofmouth M head gesture than may be the left side of the head, and thus,the left side channel). Additionally, as can be observed, during thestationary head gesture, charted sensor data 505 of the light sensordata channel of assembly 114 b of the right side of head H and chartedsensor data 506 of the light sensor data channel of assembly 114 h ofthe left side of head H may exhibit somewhat similar signalcharacteristics (e.g., as the left side channel and the right sidechannel may be similarly affected (e.g., existing in a resting pulsatilestate) by a stationary head gesture).

Whereas certain visual signal characteristics may be observed in theexemplary sensor data of FIG. 5, a number of quantitative signalcharacteristics may be calculated based on the sensor data beforeclustering. For example, an amplitude difference can be calculatedbetween a peak and a trough of the sensor data, with sign indicatingwhether the peak comes before the trough or vice versa, a timedifference can be calculated between a peak and a trough of the sensordata, a maximum amplitude can be calculated, a period between peaks ofthe sensor data can be calculated, and/or a phase can be detected in thesensor data (e.g., use phase difference of right and left sensors (e.g.,arrival time difference of pulse in left and right sensors) forphysiological data), among other possibilities. In some examples, signalcharacteristics can be observed in a frequency domain. For example, oneor more frames of sensor data may be analyzed (e.g., by a Fouriertransform) to extract frequency information as additional signalcharacteristics. These and other signal characteristics can be extractedfrom any or all of the channels of sensor data, including the channelsof light sensor data, any channels of force sensor data, any channels ofaccelerometer sensor data, and/or any channels of any other suitabletype of sensor data of system 1.

FIGS. 5A-5D illustrate two-dimensional clustering examples in accordancewith examples of the disclosure. In some examples, each frame in sensordata collection can be considered a point in multi-dimensional space,where each calculated signal characteristic for that frame may be acoordinate in the multi-dimensional space. For example, sensor data canbe collected during a first period in which a first gesture may beperformed by the user. The sensor data can be divided into a number offrames, and each frame can correspond to a set of coordinates that maybe defined by the signal characteristics calculated for that time frame.The data illustrated in FIGS. 5A-5D may represent data collected withtwo signal characteristics: (1) amplitude difference between peak andtrough for the right side light sensor data channel (e.g., the channelof each one of charted sensor data 501, 503, and 505) and (2) amplitudedifference between peak and trough for the left side light sensor datachannel (e.g., the channel of each one of charted sensor data 502, 504,and 506), for example, as discussed with respect to FIG. 5. AlthoughFIGS. 5A-5D may only show two signal characteristics, examples of thedisclosure are not so limited and contemplate using multiple kinds ofsignal characteristics from multiple channels, including light sensordata, force sensor data, sound sensor data, and/or accelerometer data,among various other possibilities. FIG. 5A may illustrate sensor data500 a collected during a first period in which a chewing head gesturemay be performed (e.g., multiple times in succession). FIG. 5B mayillustrate sensor data 500 b collected during a second period in whichan opening and closing of the right side of a user's mouth head gesturemay be being performed (e.g., multiple times in succession). FIG. 5C mayillustrate sensor data 500 c collected during a third period in which astationary head gesture may be performed. FIG. 5D may illustrate thesensor data 500 d collected during all three periods and clustered intothree clusters: first cluster 507, second cluster 508, and third cluster509. As may be shown in FIGS. 5A-5D together, most of the pointscorresponding to the first period may belong to the first cluster, mostof the points corresponding to the second period may belong to thesecond cluster, and most of the points corresponding to the third periodmay belong to the third cluster. Accordingly, it may be inferred thatany point that belongs to the first cluster 507 was collected duringperformance of a chewing head gesture, and any point that belongs to thesecond cluster 508 was collected during performance of an opening andclosing of a right side of mouth head gesture, and any point thatbelongs to the third cluster 509 was collected during performance of astationary head gesture, and that any point that does not belong to thefirst cluster 507 or the second cluster 508 or the third cluster 509 wasnot collected during performance of a chewing head gesture or an openingand closing of a right side of mouth head gesture or a stationary headgesture.

FIG. 6 is a flowchart of an illustrative process 600 for monitoring auser of a head-wearable electronic device by training a system for headgesture detection. Any suitable user interface information may bepresented to a user of system 1 (e.g., user U wearing HWD 100) in orderto prompt the user to perform a particular head gesture. For example, auser interface requesting performance of a chewing head gesture may bepresented to the user during a first period of time (e.g., visuallyand/or audibly and/or tactilely) and sensor data may be collected duringthe first period while the user interface is presented and/or duringanother period after the user interface is presented while the user mayperform the requested gesture. Additional user interfaces may bepresented to prompt a user to perform additional head gestures duringadditional periods of time to train for detection of the additionalgestures. For example, sensor data, including light sensor data, can becollected while the user performs various head gestures, for example, totrain a gesture detection algorithm. At operation 601 of process 600,during a first period in which a user performs a first head gesture(e.g., when prompted by any suitable user interface), system 1 (e.g.,HWD 100 and/or subsystem 200) may collect any suitable first sensor data(e.g., light sensor data from one, some, or each light-sensing component(e.g., photodiode) of one, some, or each light-sensing assembly 114 ofHWD 100 and/or any other suitable sensor data from any other suitablesensor component of any sensor assembly of HWD 100 and/or of subsystem200 (e.g., any pressure sensor and/or accelerometer sensor and/ormicrophone sensor and/or the like)). At operation 602, during a secondperiod in which a user performs a second head gesture (e.g., whenprompted by any suitable user interface), system 1 (e.g., HWD 100 and/orsubsystem 200) may collect any suitable second sensor data (e.g., lightsensor data from one, some, or each light-sensing component (e.g.,photodiode) of one, some, or each light-sensing assembly 114 of HWD 100and/or any other suitable sensor data from any other suitable sensorcomponent of any sensor assembly of HWD 100 and/or of subsystem 200(e.g., any pressure sensor and/or accelerometer sensor and/or microphonesensor and/or the like)). At operation 604 of process 600, any suitablefirst signal characteristic(s) may be extracted from the first sensordata collected at operation 601 (e.g., as mentioned with respect to FIG.5). At operation 606 of process 600, any suitable second signalcharacteristic(s) may be extracted from the second sensor data collectedat operation 602 (e.g., as mentioned with respect to FIG. 5). Atoperation 608 of process 600, any suitable clustering may be performedon the first signal characteristic(s) calculated at operation 604 and onthe second signal characteristic(s) calculated at operation 606 (e.g., ak-means clustering algorithm or any other suitable clusteringalgorithm). For example, at operation 610 of process 600, the clusteringalgorithm may assign one, some, or each first signal characteristic to afirst cluster of signal characteristics, and, at operation 612 ofprocess 600, the clustering algorithm may assign one, some, or eachsecond signal characteristic to a second cluster of signalcharacteristics.

It is understood that the operations shown in process 600 of FIG. 6 areonly illustrative and that existing operations may be modified oromitted, additional operations may be added, and the order of certainoperations may be altered. In some examples, system 1 (e.g., HWD 100and/or subsystem 200) can assign each cluster to one of the headgestures as part of the training process. For example, the system may beoperative to compare the first cluster to the second cluster. Then,based on comparing the first cluster to the second cluster, the systemmay be operative to determine that there are more of the first signalcharacteristics assigned to the first cluster than to the secondcluster. In accordance with such a determination that there are more ofthe first signal characteristics assigned to the first cluster than tothe second cluster, the system may be operative to assign the firstcluster to the first head gesture. Similarly, based on comparing thefirst cluster to the second cluster, the system may be operative todetermine that there are more of the second signal characteristicsassigned to the second cluster than to the first cluster. In accordancewith such a determination that there are more of the second signalcharacteristics assigned to the second cluster than to the firstcluster, the system may be operative to assign the second cluster to thesecond head gesture. In some examples, the clustering process can beseeded by initially clustering the signal characteristics based on thetime period in which the data was collected. For example, the firstcluster can be initially assigned all the signal characteristicscorresponding to the first period during which the first head gesturewas performed, and the second cluster can be initially assigned all thesignal characteristics corresponding to the second period during whichthe second head gesture was performed. Following this initialassignment, a clustering algorithm (e.g., k-means clustering) can beperformed to optimize the clusters, potentially moving some points fromthe first cluster to the second cluster, moving some points from thesecond cluster to the first cluster, and/or moving some points from thefirst and second clusters to other clusters. In some examples, thesystem may be operative to generate a template for each of the first andsecond head gestures to aid in the gesture detection process. Forexample, the system may be operative to calculate first mean signalcharacteristics for the first cluster (e.g., as part of a k-meansclustering process), and the first mean signal characteristics may beused as a template for the first cluster. Similarly, the system may beoperative to calculate second mean signal characteristics for the secondcluster (e.g., as part of a k-means clustering process), and the secondmean signal characteristics may be used as a template for the secondcluster. In another example, some or all of the first sensor data may bestored as the first template for the first cluster, and some or all ofthe second sensor data may be stored as the second template for thesecond cluster. In some examples, a generic template for each gesturemay be stored and used as a starting point for the training processbefore any user-specific data has been collected. Then, each templatecan be adjusted based on user-specific data collected during training.Any such template and/or cluster data may be stored by system 1 (e.g.,as head gesture cluster data 105 of memory assembly 104). In someexamples, additional training can be conducted to train the system todetect when the user is not performing either the first or secondgesture. The system may be operative to collect additional sensor dataduring a period in which the user does not perform the first or secondhead gesture. Signal characteristics can be calculated based on theadditional sensor data, and these signal characteristics can be assignedto a third cluster. The third cluster can be a cluster that isassociated with some third gesture (e.g., if the user performed a thirdgesture during the training period) or it can be a cluster that is notassociated with any gesture. This process may be at least partiallyrepeated for any number of gestures during any suitable time periodsduring which various external/ambient attributes/conditions may bevaried, such as a position of any ambient light source AS with respectto the user and/or a strength of any emitted ambient light AL, such thatoperation 608 may be effective for clustering signal characteristics fordifferent gestures no matter the external conditions.

FIG. 7 is a flowchart of an illustrative process 700 for monitoring auser of a head-wearable electronic device by detecting a head gesture.For example, after any suitable training or other suitable process forgenerating and/or acquiring any suitable head gesture template and/orcluster data (e.g., head gesture cluster data 105), one or more headgestures can be detected by collecting new sensor data and then usingthe clusters associated with each gesture to determine if one of thegestures has been performed. For example, at operation 701 of process700, during a third period (e.g., during use of a system after process600 has been performed (e.g., after head gesture cluster data has beenmade accessible)), system 1 (e.g., HWD 100 and/or subsystem 200) maycollect third sensor data from one, some, or each available sensorassembly (e.g., any suitable third sensor data (e.g., light sensor datafrom one, some, or each light-sensing component (e.g., photodiode) ofone, some, or each light-sensing assembly 114 of HWD 100 and/or anyother suitable sensor data from any other suitable sensor component ofany sensor assembly of HWD 100 and/or of subsystem 200 (e.g., anypressure sensor and/or accelerometer sensor and/or microphone sensorand/or the like))). At operation 704 of process 700, any suitable thirdsignal characteristic(s) may be extracted from the third sensor datacollected at operation 701 (e.g., as mentioned with respect to FIG. 5).At operation 704 of process 700, to perform gesture detection, system 1may determine whether the third signal characteristics calculated atoperation 702 belong to a first cluster (e.g., as may be defined by anyaccessible first cluster data (e.g., as may be defined at operation610)), a second cluster (e.g., as may be defined by any accessiblesecond cluster data (e.g., as may be defined at operation 612)), or athird cluster or any other cluster that may have previously beenclustered (e.g., at operation 608). The third cluster can be a clusterthat is associated with some third gesture, different from the firstgesture associated with the first cluster and different from the secondgesture associated with the second cluster, or it can be a cluster thatis not associated with any gesture. Based on determining which clusterthe third signal characteristics belong to, system 1 may detect thefirst gesture, the second gesture, or no gesture. For example, inaccordance with a determination at operation 704 that the third signalcharacteristics belong to the first cluster (e.g., a cluster associatedwith a first head gesture), system 1 may determine at operation 706 thatthe user has performed the first head gesture. In accordance with adetermination at operation 704 that the third signal characteristicsbelong to the second cluster (e.g., a cluster associated with a secondhead gesture), system 1 may determine at operation 708 that the user hasperformed the second head gesture. In accordance with a determination atoperation 704 that the third signal characteristics belong to the thirdcluster (e.g., a cluster associated with some third gesture or nogesture whatsoever), system 1 may determine at operation 710 that theuser has not performed either the first head gesture or the second headgesture.

It is understood that the operations shown in process 700 of FIG. 7 areonly illustrative and that existing operations may be modified oromitted, additional operations may be added, and the order of certainoperations may be altered. In some examples, determining whether thethird signal characteristics belong to the first cluster, the secondcluster, or the third cluster may include performing clustering (e.g., ak-means clustering algorithm, or other clustering algorithm) on thethird signal characteristics with respect to the first, second, andthird clusters. The cluster membership of the third signalcharacteristics may be determined by the results of the clustering. Insome examples, determining whether the third signal characteristicsbelong to the first cluster, the second cluster, or the third clustermay include comparing the third signal characteristics to first, second,and/or third templates corresponding to the first, second, and thirdclusters, respectively. The system may thereby be operative to determinewhether the third signal characteristics are closer to the first clusteror the second cluster based on the templates. For example, if eachtemplate includes mean signal characteristics, then the system may beoperative to calculate a first distance from the third signalcharacteristics to the first template (e.g., the first mean signalcharacteristics of the first cluster) and calculate a second distancefrom the third signal characteristics to the second template (e.g., thesecond mean signal characteristics of the second cluster). As just oneexample, the distance calculation can be a Euclidean distancecalculation between two points in multi-dimensional space. In accordancewith a determination that the first distance is shorter than the seconddistance, the system may be operative to determine that the third signalcharacteristics belong to the first cluster. In accordance with adetermination that the second distance is shorter than the firstdistance, the system may be operative to determine that the third signalcharacteristics belong to the second cluster. In some examples, thesystem may also be operative to compare the third signal characteristicsto a third template in the same manner, or, if both the first and seconddistances are longer than a predetermined threshold distance, the systemmay be operative to determine that the third signal characteristicsbelong to a third cluster by default. Based on determining which clusterthe third signal characteristics belong to, the system may then beoperative to detect the first gesture, the second gesture, or no gesture(e.g., at operations 706, 708, and/or 710). After detecting the firstgesture or the second gesture, the system may be operative to perform anoperation associated with the detected gesture. For example, if thesystem detects the first gesture, the system may be operative to performan operation in response, such as opening an application, closing anapplication, returning to a home screen, messaging a contact, adjustingan audio output volume, and/or any other suitable functionality (e.g.,system 1 may determine and share a determined head gesture as at least aportion of sensor mode data 324 with at least one managed element 390 ofsystem 1 (e.g., of device 100 and/or of any suitable subsystem 200 ofsystem 1) at least partially based on the received sensor state data 322(e.g., third signal characteristics), where such sensor mode data 324may be received by managed element 390 for controlling at least onecharacteristic of managed element 390). In some examples, sensor data(e.g., the first, second, or third sensor data described above) can befurther processed before extracting signal characteristics (e.g., thefirst, second, or third signal characteristics described above). Forexample, a band pass filter may be applied to sensor data to filter outheart rate frequencies from the sensor data. As light sensor data mayvary according to the periodic motion of blood through human headtissue, it may be beneficial to filter out these frequencies to betterisolate the contribution of head gesture motion to the signalcharacteristics for detecting a particular head gesture (although, suchperiodic blood motion frequencies may be used for detecting any suitableheart rate characteristic(s) of the user).

The above provides just a few examples as to how head gesture templateand/or cluster data (e.g., head gesture cluster data 105) may beobtained and/or used to determine an appropriate head gesture of a userwearing an HWD using light sensor data. For example, a head gesturemodel may be developed and/or generated (e.g., as head gesture data 105)for use in evaluating and/or predicting and/or estimating and/ordetermining a particular head gesture for a particular type of HWD on ageneral and/or a particular head (e.g., for an experiencing entity(e.g., a particular user or a particular subset or type of user or allusers generally (e.g., using a particular type of HWD)). For example, ahead gesture model may be a learning engine for any experiencing entity,such as for a particular HWD type and for any general user and/or for aparticular user (e.g., with a particular head shape and/or particulargesture mannerisms), where the learning engine may be operative to useany suitable machine learning to use certain sensor data (e.g., one ormore various types or categories of sensor data that may be detected byany suitable sensor assembly(ies) of the HWD and/or of any suitablepaired subassembly(ies) (e.g., one, some, or each light sensor channeldata of one, some, or each of light-sensing assemblies 114 a-114 i, anyother suitable sensor channel data of one, some, or each oflight-sensing assemblies 114 a-114 i and/or of subsystem 200)) in orderto predict, estimate, and/or otherwise determine a current head gestureof the user. For example, the learning engine may include any suitableneural network (e.g., an artificial neural network) that may beinitially configured, trained on one or more sets of sensor data thatmay be generated during the performance of one or more known headgestures, and then used to predict a particular head gesture based onanother set of sensor data. A neural network or neuronal network orartificial neural network may be hardware-based, software-based, or anycombination thereof, such as any suitable model (e.g., an analyticalmodel, a computational model, etc.), which, in some embodiments, mayinclude one or more sets or matrices of weights (e.g., adaptive weights,which may be numerical parameters that may be tuned by one or morelearning algorithms or training methods or other suitable processes)and/or may be capable of approximating one or more functions (e.g.,non-linear functions or transfer functions) of its inputs. The weightsmay be connection strengths between neurons of the network, which may beactivated during training and/or prediction. A neural network maygenerally be a system of interconnected neurons that can compute valuesfrom inputs and/or that may be capable of machine learning and/orpattern recognition (e.g., due to an adaptive nature). A neural networkmay use any suitable machine learning techniques to optimize a trainingprocess. The neural network may be used to estimate or approximatefunctions that can depend on a large number of inputs and that may begenerally unknown. The neural network may generally be a system ofinterconnected “neurons” that may exchange messages between each other,where the connections may have numeric weights (e.g., initiallyconfigured with initial weight values) that can be tuned based onexperience, making the neural network adaptive to inputs and capable oflearning (e.g., learning pattern recognition). A suitable optimizationor training process may be operative to modify a set of initiallyconfigured weights assigned to the output of one, some, or all neuronsfrom the input(s) and/or hidden layer(s). A non-linear transfer functionmay be used to couple any two portions of any two layers of neurons,including an input layer, one or more hidden layers, and an output(e.g., an input to a hidden layer, a hidden layer to an output, etc.).Different input neurons of the neural network may be associated withrespective different types of sensor data categories and may beactivated by sensor data of the respective sensor data categories (e.g.,light sensor channel data for each possible light sensor channel of eachlight-sensing assembly of the HWD, additional sensor channel data foreach possible additional sensor channel of the HWD (e.g., sound data,motion data, force data, temperature data, ambient color/white pointchromaticity data, geo-location data, time data, location type, time ofday, day of week, week of month, week of year, month of year, season,holiday, time zone, and/or the like), any suitable data indicative of anactivity of the user (e.g., exercising, gaming/viewing (e.g., currentstatus of a currently played media), sleeping, working, reading, etc.),and/or the like may be associated with one or more particular respectiveinput neurons of the neural network and sensor category data for theparticular sensor category may be operative to activate the associatedinput neuron(s)). The weight assigned to the output of each neuron maybe initially configured (e.g., at operation 802 of process 800 of FIG.8) using any suitable determinations that may be made by a custodian orprocessor (e.g., device 100 and/or auxiliary subsystem 200) of the headgesture or sensor model (e.g., head gesture data 105) based on the dataavailable to that custodian.

The initial configuring of the learning engine or head gesture model forthe experiencing entity (e.g., the initial weighting and arranging ofneurons of a neural network of the learning engine) may be done usingany suitable data accessible to a custodian of the head gesture model(e.g., a manufacturer of device 100 or of a portion thereof (e.g., amodel 105 m of head gesture data 105), any suitable maintenance entitythat manages auxiliary subsystem 200, and/or the like), such as dataassociated with the configuration of other learning engines of system 1(e.g., learning engines or head gesture models for similar experiencingentities), data associated with the experiencing entity (e.g., initialbackground data accessible by the model custodian about the experiencingentity's composition, size, shape, age, any suitable biometricinformation, background, interests, goals, past experiences, and/or thelike), data assumed or inferred by the model custodian using anysuitable guidance, and/or the like. For example, a model custodian maybe operative to capture any suitable initial background data about theexperiencing entity in any suitable manner, which may be enabled by anysuitable user interface provided to an appropriate subsystem or deviceaccessible to one, some, or each experiencing entity (e.g., a model appor website). The model custodian may provide a data collection portalfor enabling any suitable entity to provide initial background data forthe experiencing entity. The data may be uploaded in bulk or manuallyentered in any suitable manner. In a particular embodiment where theexperiencing entity is a particular user or a group of users, thefollowing is a list of just some of the one or more potential types ofdata that may be collected by a model custodian (e.g., for use ininitially configuring the model): sample questions for which answers maybe collected may include, but are not limited to, questions related toan experiencing entity's age, head shape, comfort level while wearing anHWD, evaluation of perceived or otherwise measured gesture (e.g.,gesture and/or motion and/or action and/or vocalization and/or emotionand/or thought and/or brain function and/or heart rate characteristicand/or other biometric characteristic (e.g., as predicted by the modelusing detected by HWD sensor data)) with respect to a particularpreviously intended and/or conducted and/or performed gesture (e.g.,gesture and/or motion and/or action and/or vocalization and/or emotionand/or thought and/or brain function and/or heart rate characteristicand/or other biometric characteristic (e.g., as indicated by theexperiencing entity through selection of one gesture from a list ofgestures provided for selection (e.g., in a survey))), and/or the like.

A head gesture model custodian may receive from the experiencing entity(e.g., at operation 804 of process 800 of FIG. 8) not only HWD sensorcategory data for at least one HWD sensor category for a gesture thatthe experiencing entity is currently experiencing conducting or carryingout or undergoing or has previously experienced or conducted or carriedout or undergone, but also a score for that gesture experience (e.g., ascore that the experiencing entity and/or a non-HWD sensor (e.g., sensorassembly 214 of subsystem 200) may supply as an indication of thegesture that the experiencing entity experienced from experiencing thegesture). This may be enabled by any suitable user interface provided toany suitable experiencing entity by any suitable head gesture modelcustodian (e.g., a user interface app or website that may be accessed bythe experiencing entity). The head gesture model custodian may provide adata collection portal for enabling any suitable entity to provide suchdata. The score (e.g., head gesture score) for the gesture may bereceived and may be derived from the experiencing entity in any suitablemanner. For example, a single questionnaire or survey may be provided bythe model custodian for deriving not only experiencing entity responseswith respect to HWD sensor category data for a gesture, but also anexperiencing entity score for the gesture. The model custodian may beconfigured to provide best practices and standardize much of theevaluation, which may be determined based on the experiencing entity'sgoals and/or objectives as captured before the gesture may have beenexperienced. In some embodiments, in order to train one or more models,a user may manually or actively provide information to the system thatis indicative of one or more gestures known by the user to have beencarried out by the user, where such information may be used to defineone or more outputs of one or more models (e.g., information indicativeof a particular gesture that the user intentionally carried out whileHWD sensor data was collected by the system, such as chewing, blinking,winking, smiling, eyebrow raising, humming or other internalvocalizations (e.g., “mmm-hmm”, “uh-huh”, etc.), inaudible cues, jawmotions, speaking or other external explicit language vocalization,mouth opening (e.g., full mouth opening, left-side mouth opening,right-side mouth opening, etc.), ear wiggling, smirking, smiling,frowning, grimacing, cheek motioning, removing the HWD from the user'shead, adorning the user's head with the HWD, and/or the like).Additionally or alternatively, in order to train one or more models, oneor more non-HWD sensing components may be used to provide information tothe system that is indicative of one or more gestures known to have beencarried out by the user, where such information may be used to defineone or more outputs of one or more models (e.g., information indicativeof a particular gesture carried out while HWD sensor data was collectedby the system, such as one or more particular biometric characteristicgestures of the user as may be detected by any suitable sensingcomponent(s) of sensor assembly 214 of any suitable subsystem 200 (e.g.,a dedicated biometric sensing subsystem (e.g., a dedicated PPG, fNIRspectroscope, EEG machine, etc.))).

A learning engine or head gesture model for an experiencing entity maybe trained (e.g., at operation 806 of process 800 of FIG. 8) using thereceived HWD sensor category data for the gesture (e.g., as inputs of aneural network of the learning engine) and using the received score forthe gesture (e.g., as an output of the neural network of the learningengine). Any suitable training methods or algorithms (e.g., learningalgorithms) may be used to train the neural network of the learningengine, including, but not limited to, Back Propagation, ResilientPropagation, Genetic Algorithms, Simulated Annealing, Levenberg,Nelder-Meade, and/or the like. Such training methods may be usedindividually and/or in different combinations to get the bestperformance from a neural network. A loop (e.g., a receipt and trainloop) of receiving HWD sensor category data and a score for a gestureand then training the head gesture model using the received HWD sensorcategory data and score (e.g., a loop of operation 804 and operation 806of process 800 of FIG. 8) may be repeated any suitable number of timesfor the same experiencing entity and the same learning engine for moreeffectively training the learning engine for the experiencing entity,where the received HWD sensor category data and the received scorereceived of different receipt and train loops may be for differentgestures or for the same gesture (e.g., at different times) and/or maybe received from the same source or from different sources of theexperiencing entity (e.g., from different users of the same or a similarHWD) (e.g., a first receipt and train loop may include receiving HWDsensor category data and a score from a first user of a first age withrespect to that user's experience with a first (e.g., intended) gesture,while a second receipt and train loop may include receiving HWD sensorcategory data and a score from a second user of a second age withrespect to that user's experience with the first (e.g., intended)gesture, while a third receipt and train loop may include receiving HWDsensor category data and a score from a third user of the first age withrespect to that user's experience with a second (e.g., intended)gesture, and/or the like), while the training of different receipt andtrain loops may be done for the same learning engine using whatever HWDsensor category data and score was received for the particular receiptand train loop. The number and/or type(s) of the one or more HWD sensorcategories for which HWD sensor category data may be received for onereceipt and train loop may be the same or different in any way(s) thanthe number and/or type(s) of the one or more HWD sensor categories forwhich HWD sensor category data may be received for a second receipt andtrain loop.

A head gesture model custodian may access (e.g., at operation 808 ofprocess 800 of FIG. 8) HWD sensor category data for at least one HWDsensor category for another gesture (e.g., another intended gesture)that is different than any intended gesture considered at any HWD sensorcategory data receipt of a receipt and train loop for training thelearning engine for the experiencing entity). In some embodiments, thisother gesture may be a gesture that has not been specificallyexperienced by any experiencing entity prior to use of the gesture modelin an end user use case. Although, it is to be understood that thisother gesture may be any suitable gesture. The HWD sensor category datafor this other gesture may be accessed from or otherwise provided by anysuitable source(s) using any suitable methods (e.g., from one or moresensor assemblies and/or input assemblies of any suitable device(s) 100and/or subsystem(s) 200 that may be associated with (e.g., worn by theuser carrying out) the particular gesture at the particular time) foruse by the gesture model custodian (e.g., processor assembly 102 ofdevice 100).

This other gesture (e.g., gesture of interest) may then be scored (e.g.,at operation 810 of process 800 of FIG. 8) using the learning engine orgesture model for the experiencing entity with the HWD sensor categorydata accessed for such another gesture. For example, the HWD sensorcategory data accessed for the gesture of interest may be utilized asinput(s) to the neural network of the learning engine (e.g., atoperation 810 of process 800 of FIG. 8) similarly to how the HWD sensorcategory data accessed at a receipt portion of a receipt and train loopmay be utilized as input(s) to the neural network of the learning engineat a training portion of the receipt and train loop, and suchutilization of the learning engine with respect to the HWD sensorcategory data accessed for the gesture of interest may result in theneural network providing an output indicative of a gesture score orgesture level or gesture state that may represent the learning engine'spredicted or estimated gesture to have been experienced by theexperiencing entity.

After a gesture score (e.g., any suitable gesture state data (e.g.,gesture state data or user state data or sensor state data 322 of FIG.3)) is determined (e.g., estimated or predicted by the model) for agesture of interest (e.g., for a current gesture being experienced by anexperiencing entity (e.g., for a particular time and/or during aparticular activity)), it may be determined (e.g., at operation 812 ofprocess 800 of FIG. 8) whether the realized gesture score satisfies aparticular condition of any suitable number of potential conditions,and, if so, the model custodian or any other suitable processor assemblyor otherwise (e.g., of device 100) may generate any suitable controldata (e.g., sensor mode data (e.g., sensor mode data 324 of system 301of FIG. 3)) that may be associated with that satisfied condition forcontrolling any suitable functionality of any suitable assembly ofdevice 100 or of device 200 or otherwise (e.g., for adjusting a userinterface presentation to a user (e.g., to present an indication of theuser's heart rate to the user when the satisfied condition is indicativeof a heart rate above a certain threshold) and/or for activating acamera functionality (e.g., to capture the environment of the user whenthe satisfied condition is indicative of the user being scared (e.g.,when the satisfied condition is indicative of the user having gasped))and/or for controlling any suitable functionality of any suitable sensorassembly of device 100 or otherwise (e.g., for turning on or off aparticular type of sensor and/or for adjusting the functionality (e.g.,the accuracy) of a particular type of sensor (e.g., to gather anyadditional suitable sensor data)), and/or the like). A gesture score maybe indicative of a probability of one or more gestures having beenintended or carried out (e.g., voluntarily and/or involuntarily) orendured by the experiencing entity and/or of a characteristic of one ormore gestures. For example, a score may be indicative of 90% likelihoodthat the user gasped and indicative of a heart rate between X and Y or aheart rate of Z. As just one other example, a score may be indicative of83% likelihood that the user smiled and indicative of a heart ratevariability between G and H or a heart rate variability of I. In someembodiments, a first model may be trained and later used to score afirst type of gesture (e.g., likelihood of a gasp) while a second modelmay be trained and later used to score a second type of gesture (e.g.,value or range of a heart rate). Certain types or all types of HWDsensor data for a particular moment may be provided as inputs to certainones or to each available gesture model, such that various models mayeach provide a respective output score for that moment, where eachoutput score may be analyzed with respect to one or more differentrespective conditions depending on the type of model providing theoutput score. For example, all various HWD sensor data generated when auser acts a certain way during a certain moment may be provided asinputs to one or more different models, each of which may generate adifferent output score, each of which may be compared to one or moredifferent conditions, for determining one or more gestures or gestureconditions most likely to have been carried out or endured orexperienced by the user during that moment. A certain condition may bedefined by a certain threshold (e.g., a determined likelihood of agasping gesture being at least 90% or a determined heart rate being atleast a value X, etc.) above which the predicted gesture score ought toresult in a warning or other suitable instruction or adjustedfunctionality being provided to the experiencing entity. A thresholdscore or condition may be defined or otherwise determined (e.g.,dynamically) in any suitable manner and may vary between differentexperiencing entities and/or between different gestures of interestand/or between different combinations of such experiencing entities andgestures and/or in any other suitable manner.

If a gesture of interest is experienced by the experiencing entity, thenany suitable gesture behavior data (e.g., any suitable user behaviorinformation), which may include an experiencing entity provided gesturescore (e.g., I 100% gasped), may be detected during that experience andmay be stored (e.g., along with any suitable gesture characteristicinformation of that gesture) as gesture behavior data and/or may be usedin an additional receipt and train loop for further training thelearning engine. Moreover, in some embodiments, a gesture modelcustodian may be operative to compare a predicted gesture score for aparticular gesture of interest with an actual experiencing entityprovided gesture score for the particular gesture of interest that maybe received after or while the experiencing entity may be actuallyexperiencing the gesture of interest and enabled to actually score thegesture of interest (e.g., using any suitable user behavior information,which may or may not include an actual user provided score feedback).Such a comparison may be used in any suitable manner to further trainthe learning engine and/or to specifically update certain features(e.g., weights) of the learning engine. For example, any algorithm orportion thereof that may be utilized to determine a gesture score may beadjusted based on the comparison. A user (e.g., experiencing entity(e.g., an end user of device 100)) may be enabled by the gesture modelcustodian to adjust one or more filters, such as a profile of gesturesthey prefer to or often experience and/or any other suitable preferencesor user profile characteristics (e.g., age, weight, seeing ability,etc.) in order to achieve such results. This capability may be usefulbased on changes in an experiencing entity's capabilities and/orobjectives as well as the gesture score results. For example, if a userloses its ability to hear or see color, this information may be providedto the model custodian, whereby one or more weights of the model may beadjusted such that the model may provide appropriate scores in thefuture.

Therefore, any suitable gesture model custodian may be operative togenerate and/or manage any suitable gesture model or gesture learningengine that may utilize any suitable machine learning, such as one ormore artificial neural networks, to analyze certain gesture data (e.g.,HWD sensor data) of a performed or detected gesture to predict/estimatethe gesture score or intended gesture of that performed gesture for aparticular user (e.g., generally, and/or at a particular time, and/orwith respect to one or more planned activities), which may enableintelligent suggestions be provided to the user and/or intelligentsystem functionality adjustments be made for improving the user'sexperiences. For example, a gesture engine may be initially configuredor otherwise developed for an experiencing entity based on informationprovided to a model custodian by the experiencing entity that may beindicative of the experiencing entity's specific preferences fordifferent gestures and/or gesture types (e.g., generally and/or forparticular times and/or for particular planned activities) and/or of theexperiencing entity's specific experience with one or more specificgestures. An initial version of the gesture engine for the experiencingentity may be generated by the model custodian based on certainassumptions made by the model custodian, perhaps in combination withsome limited experiencing entity-specific information that may beacquired by the model custodian from the experiencing entity prior tousing the gesture engine, such as the experiencing entity's age,language(s) spoken, hair color, any suitable biometric characteristics,and/or the like. The initial configuration of the gesture engine may bebased on data for several HWD sensor categories, each of which mayinclude one or more specific HWD sensor category data values, each ofwhich may have any suitable initial weight associated therewith, basedon the information available to the model custodian at the time ofinitial configuration of the engine (e.g., at operation 802 of process800 of FIG. 8). As an example, an HWD sensor category may be forcedetected by a force sensor (e.g., AS1 component 164), and the variousspecific HWD sensor category data values for that HWD sensor categorymay include any force less than A force, any force between B force and Cforce, any force between C force and D force, and/or the like, each ofwhich may have a particular initial weight associated with it. Asanother example, an HWD sensor category may be amount of light type XYZdetected by a specific light-sensing component of a specific HWD (e.g.,an amount of IR light detected by PD2 component 134 of device 100 (e.g.,the total amount of IR light detected by that PD2 component 134, a ratioof the total amount of IR light detected by that PD2 component 134compared to the total amount of IR light emitted by LE1 component 154for potential detection by PD2 component 134, or the like)). Forexample, each channel of light sensor data available to the HWD may berepresented by its own HWD sensor category, and the amount of lightdetected by that channel may be used to define the HWD sensor categorydata for that channel's HWD sensor category.

Once an initial gesture engine has been created for an experiencingentity, the model custodian may provide a survey to the experiencingentity that asks for specific information with respect to a particulargesture that the experiencing entity has experienced in the past orwhich the experiencing entity is currently experiencing. Not only may asurvey ask a user for or otherwise (e.g., automatically) obtainobjective information about a particular gesture, such as anidentification of the location at which the gesture was performed, thetime at which the gesture was experienced, the current sleep level ofthe experiencing entity, the current nutrition level of the experiencingentity, the current mindfulness level of the experiencing entity, anactivity performed by the experiencing entity while experiencing thegesture (e.g., playing a video game, reading a book, talking on thetelephone, watching a sporting event, etc.), the heart rate or otherbiometric characteristic of the user (e.g., as determined by a non-HWDsensor), and/or the like, but also for subjective information about thegesture, such as the experiencing entity's intended or known to beperformed gesture (e.g., I 100% yawned, I 100% gasped, I was 80% scaredand 20% happy, etc.) and/or the like. Each completed experiencing entitysurvey for one or more gestures (e.g., one or more gestures generallyand/or for one or more times and/or for one or more concurrentactivities) by one or more particular experiencing entity respondents ofthe experiencing entity may then be received by the model custodian andused to train the gesture engine. By training the gesture engine withsuch experiencing entity feedback on one or more prior and/or currentgesture experiences, the gesture engine may be more customized to theexperiencing entity by adjusting the weights of one or more categoryoptions to an updated set of weights for providing an updated gestureengine.

It is to be understood that device 100 and/or any other device orsubsystem available to system 1 (e.g., any remote subsystem via theinterne or any other suitable network) may be a model custodian for atleast a portion or all of one or more gesture models (e.g., of gesturedata 105). A particular model (e.g., a particular one of one or moregesture models 105 m of gesture data 105) may be for one or moreparticular users and/or one or more particular HWDs and/or one or moreparticular gestures.

To accurately determine a head gesture of a user of HWD 100, anysuitable portion of system 1, such as device 100, may be configured, touse various information sources in combination with any available headgesture data 105 (e.g., any suitable one or more gesture models) inorder to classify or predict a current head gesture of the user. Forexample, any suitable processing circuitry or assembly (e.g., a sensormodule) of device 100 may be configured to gather and to process varioustypes of sensor data, in conjunction with head gesture data 105, todetermine what type of head gesture has been performed or is beingperformed by the user. For example, any suitable sensor data from one ormore of any or each sensor assembly 114 of device 100, with or withoutany suitable sensor data from auxiliary environment subsystem 200, andany application data of any application 103 being run by device 100 maybe utilized in conjunction with any suitable head gesture data, such aswith a gesture model 105 m of head gesture data 105, to determine a headgesture of the user efficiently and/or effectively.

FIG. 3 shows a schematic view of a sensor management system 301 of HWD100 that may be provided to manage sensor states of HWD 100 (e.g., todetermine a head gesture of a user wearing HWD 100 and to manage a modeof operation of HWD 100 and/or of any other suitable subsystem (e.g.,subsystem 200) of system 1 based on the determined sensor state). Inaddition to or as an alternative to using any device sensor data 114 dthat may be generated by any suitable sensor data channel(s) of anysuitable sensing assemblies 114 (e.g., as may be automaticallytransmitted to sensor management system 301 and/or received by sensormanagement system 301 in response to device sensor request data 114 r),sensor management system 301 may use various other types of dataaccessible to device 100 in order to determine a current sensor state ofsystem 1 (e.g., in conjunction with one or more gesture models 105 m ofhead gesture data 105), such as any suitable data provided by one ormore of auxiliary subsystems 200 (e.g., data 91 from one or moreassemblies of auxiliary subsystem 200), an activity application 103 ofdevice 100 (e.g., data 103 d that may be provided by an activityapplication 103 (e.g., automatically and/or in response to request data103 r) and that may be indicative of one or more current activities ofthe user (e.g., current state of a video game being played by the user,type of movie being watched by the user, type of book being read byuser, etc.). In response to determining the current sensor state (e.g.,at least a recent head gesture performed by the user), sensor managementsystem 301 may apply at least one sensor-based mode of operation to atleast one managed element 390 (e.g., any suitable assembly of device 100and/or any suitable assembly of subsystem 200 or otherwise of system 1)based on the determined sensor state (e.g., to suggest certain userbehavior and/or to control the functionality of one or more systemassemblies) for improving a user's experience. For example, as shown inFIG. 3, sensor management system 301 may include a sensor module 340 anda management module 380.

Sensor module 340 of sensor management system 301 may be configured touse various types of data accessible to HWD 100 in order to determine(e.g., characterize) a sensor state (e.g., a current head gesture of auser of HWD 100 with or without any other characteristic(s) (e.g., heartrate, etc.)). As shown, sensor module 340 may be configured to receiveany suitable device sensor data 114 d that may be generated and sharedby any suitable device sensor assembly 114 when HWD 100 is worn on headH of user U (e.g., automatically or in response to any suitable requesttype of device sensor request data 114 r that may be provided to anysensor assembly 114), any suitable auxiliary subsystem data 91 that maybe generated and shared by any suitable auxiliary subsystemassembly(ies) based on any sensed data or any suitable auxiliarysubsystem assembly characteristics (e.g., automatically or in responseto any suitable request type of auxiliary subsystem data 99 that may beprovided to auxiliary subsystem 200), any suitable activity applicationstatus data 103 d that may be generated and shared by any suitableactivity application 103 that may be indicative of one or more useractivities (e.g., automatically or in response to any suitable requesttype of activity application request data 103 r that may be provided toactivity application 103), and sensor module 340 may be operative to usesuch received data in any suitable manner in conjunction with anysuitable head gesture model data and/or any suitable head gesturecluster data (e.g., any suitable gesture model(s) 105 m of head gesturedata 105) to determine any suitable sensor state (e.g., with headgesture data 105 d that may be any suitable portion or the entirety ofhead gesture data 105, which may be accessed automatically and/or inresponse to any suitable request type of head gesture request data 105 rthat may be provided to a provider of head gesture data 105 (e.g.,memory assembly 104 or a memory assembly of auxiliary subsystem 200)).Any suitable portions of one or more of data 114 d, data 91, and data103 d may be used as category data inputs for one or more models of data105 d.

Once sensor module 340 has determined a current sensor state for a userof HWD 100 (e.g., based on any suitable combination of one or more ofany suitable received data 114 d, 91, 103 d, and 105 d), sensor module340 may be configured to generate and transmit sensor state data 322 tomanagement module 380, where sensor state data 322 may be indicative ofat least one determined sensor state for the user of HWD 100 (e.g., oneor more of a current head gesture, current heart rate, current speed,current location, current emotion, etc.). In response to determining asensor state or one or more gestures of a user of HWD 100 by receivingsensor state data 322, management module 380 may be configured to applyat least one sensor-based mode of operation to at least one managedelement 390 of system 1 based on the determined sensor state. Forexample, as shown in FIG. 3, sensor management system 301 may includemanagement module 380, which may be configured to receive sensor statedata 322 from sensor module 340, as well as to generate and share sensormode data 324 with at least one managed element 390 of system 1 (e.g.,of HWD 100 and/or of any other suitable subsystem 200) at leastpartially based on the received sensor state data 322, where such sensormode data 324 may be received by managed element 390 for controlling atleast one characteristic of managed element 390. Managed element 390 maybe any suitable assembly of device 100 (e.g., any processor assembly102, any memory assembly 104 and/or any data stored thereon, anycommunications assembly 106, any power supply assembly 108, any inputassembly 110, any output assembly 112, any sensor assembly 114, etc.)and/or any suitable assembly of any suitable auxiliary environmentsubsystem 200 of system 1, and sensor mode data 324 may control managedelement 390 in any suitable way, such as by enhancing, enabling,disabling, restricting, and/or limiting one or more certainfunctionalities associated with such a managed element (e.g.,controlling motor 122 to better position light-sensing assembly 114 bfor more effective sensing (e.g., due to ambient light, insufficientstrength of contact with user's head, etc.), turning on a videorecording capability of device 100 or subsystem 200 (e.g., due todetecting a user gasp gesture and/or a user scared gesture), and/or thelike).

Sensor mode data 324 may be any suitable device control data forcontrolling any suitable functionality of any suitable assembly of HWD100 as a managed element 390 (e.g., any suitable device output controldata for controlling any suitable functionality of any suitable outputassembly 112 of device 100 (e.g., for adjusting a user interfacepresentation to user U (e.g., to provide a suggestion or an indicationof any suitable sensor data (e.g., heart rate))), and/or any suitabledevice sensor control data (e.g., a control type of device sensorrequest data 114 r) for controlling any suitable functionality of anysuitable sensor assembly 114 of device 100 (e.g., for turning on or offa particular type of sensor and/or for adjusting the functionality(e.g., the accuracy) of a particular type of sensor (e.g., to gather anyadditional suitable sensor data)), and/or any suitable activityapplication control data (e.g., a control type of activity applicationrequest data 103 r) for updating or supplementing any input dataavailable to activity application 103 that may be used to determine acurrent activity, and/or the like). Additionally or alternatively,sensor mode data 324 may be any suitable auxiliary subsystem data 99 forcontrolling any suitable functionality of any suitable auxiliarysubsystem 200 as a managed element 390 (e.g., capture a photograph orturn on a video recording functionality of subsystem 200 in response todetecting a particular gesture (e.g., in response to detection of a usergasping or an increase in heart rate (e.g., for security purposes))).Additionally or alternatively, sensor mode data 324 may be any suitablehead gesture update data (e.g., an update type of head gesture requestdata 105 r) for providing any suitable data to head gesture data 105 asa managed element 390 (e.g., any suitable head gesture update data forupdating a model or cluster of head gesture data 105 (e.g., a model 105m) in any suitable manner).

FIG. 8 is a flowchart of an illustrative process 800 for monitoring auser of a head-wearable electronic device. At operation 802 of process800, a head gesture model custodian (e.g., a gesture model custodiansystem) may initially configure a learning engine (e.g., gesture model105 m) for an experiencing entity. At operation 804 of process 800, thehead gesture model custodian may receive, from the experiencing entity,HWD sensor category data for at least one HWD sensor category for agesture and a score for the gesture. At operation 806 of process 800,the head gesture model custodian may train the learning engine using thereceived HWD sensor category data and the received score. At operation808 of process 800, the head gesture model custodian may access HWDsensor category data for the at least one HWD sensor category foranother gesture. At operation 810 of process 800, the head gesture modelcustodian may score the other gesture, using the learning engine, withthe accessed HWD sensor category data for the other gesture. Atoperation 812 of process 800, when the score for the other gesturesatisfies a condition, the head gesture model custodian may generatecontrol data associated with the satisfied condition.

It is understood that the operations shown in process 800 of FIG. 8 areonly illustrative and that existing operations may be modified oromitted, additional operations may be added, and the order of certainoperations may be altered.

An output score of a model for a particular gesture type or any othersuitable determination or estimation of the likelihood of a particulargesture being detected based on certain types or all types of HWD sensordata for a particular moment (e.g., a score of operation 810 and/or adetermination of one of operations 706 or 708 and/or a sensor state ofsensor state data 322) may be combined with such a determination orestimation of the likelihood of one, some, or each other particulargesture being detected based on certain types or all types of HWD sensordata for a particular moment (e.g., another score of another iterationof operation 810 and/or a determination of one of another iteration ofoperations 706 or 708 and/or another sensor state of other sensor statedata 322), and the combination of such determinations or estimations oflikelihood for any suitable number of gestures for a particular moment(e.g., concurrently detected or immediately sequentially detectedgestures or likelihoods thereof) may be used to make any suitablecombined determination, such as a determination or estimation as to theuser's state of being, which may then be used to control managed element390 in any suitable manner. For example, such various determinations orestimations of likelihood for any suitable number of gestures themselvesmay be inputs to one or more secondary models and/or may be used toperform clustering or otherwise to provide an output score of a modelfor a particular user's state of being or any other suitabledetermination or estimation of the likelihood of a particular user'sstate of being based on such inputs. These inputs can becross-referenced with respect to any suitable machine learning supersetand tied to best a fit profile based on similar or baseline users. Anysuitable particular types of user's state of being may be determined,including, but not limited to, a determination of one's physical and/orpsychological state with regard to attentiveness, receptiveness,alertness, drowsiness, boredom, stimulation, confidence, deception(e.g., a user is lying), anxiety, depression, worry, serenity, degree ofrelaxation, degree of stimulation, mental state, and/or the like. Forexample, a combination of 80% likelihood of smiling and a 90% chance ofa regular heart rate may result in a 90% likelihood of a serenity stateof being. As another example, a combination of 80% likelihood of a wideopen mouth and a 75% likelihood of raised eyebrows and a 90% chance of ahigh heart rate may result in a 90% likelihood of an anxious state ofbeing.

The system may be configured to use sequential signals to differentiatebetween different states of being. For example, the system may beconfigured to use sequential signals to differentiate between scared orhappy or surprised. For example, in a scenario where a user comes hometo a dark house and turns on the lights and then (1) the user's eyeswiden involuntarily, the eyebrows move up, the ears move back, and then(2a) the user either smiles (e.g., when the user spots his dog on thecouch) or (2b) the user gasps (e.g., when the user sees a burglar) forpotentially determining happy or scared or surprised. The system may beconfigured to monitor a series of rich facial gestures including anysuitable numerous involuntary, unnoticed, facial movements that may betracked to determine any suitable state(s) of being, which may be usedto control the system in any suitable way(s). For example, in responseto detecting a likelihood of a deception state of being where the usermay be lying, the system may be operative to communicate this to anysuitable entity (e.g., as a lie detector test). For example, in responseto detecting a likelihood of a drowsiness state of being where the usermay be falling asleep, the system may be operative to generate hapticfeedback for attempting to stimulate the user (e.g., for encouraging auser to focus if becoming drowsy or daydreaming while in class or forinstructing the user to memorialize a thought if the user is determinedto be daydreaming. For example, in response to detecting a likelihood ofa frowning state of being where the user may be expressing signs ofdispleasure, the system may be operative to generate haptic feedback forattempting to notify the user (e.g., discreet feedback for notifying theuser to look more cheerful if determined to be displeased at aninopportune situation (e.g., during a job interview)).

FIG. 9 is a flowchart of an illustrative process 900 for monitoring auser of a head-wearable electronic device. At operation 902 of process900, light sensor data may be obtained for one, some, or each channel oflight sensor data from one, some, or each light-sensing component ofone, some, or each light-sensing assembly of a head-wearable electronicdevice (e.g., light sensor data from each channel of each light-sensingcomponent 124/134/144 of each light-sensing 114 of HWD 100 (e.g., at aparticular moment in time or for a particular duration of time)).

At operation 904 of process 900, a functional proximity for one, some,or each light-sensing component of the HWD may be determined (e.g.,using at least one additional-sensing component associated with thelight-sensing component). For example, at least one ofadditional-sensing components 164 and 174 of sensing assembly 114 b maybe associated with (e.g., positioned adjacent or otherwise close to)one, some, or each of light-sensing components 124, 134, and/or 144, andsuch an additional-sensing component may be operative to provide sensordata indicative of the proximity of that additional-sensing component(and, thereby, of its associated light-sensing component(s)) to asurface (e.g., skin surface HSs) against which the light-sensingcomponent(s) may function. As just one example, additional-sensingcomponent 174 may be a force or contact or pressure sensor or any othersuitable sensor that may be operative to provide functional proximitydata that may be indicative of the proximity of its associatedlight-sensing component(s) to a functional surface of the user's head,where such functional proximity data may be used (e.g., by any suitableprocessor of system 1) to determine whether or not the associatedlight-sensing component(s) are functionally proximate the functionalsurface in order to determine whether the associated light sensor datamay be relied upon (e.g., at all or with a particular weight) or whetherthe associated light sensor data ought to be disregarded (e.g., atoperation 910). For example, a light-sensing component held against auser's skin surface with a particular force or pressure or within aparticular range thereof may provide more reliable light sensor datathan a light-sensing component determined to be held at a distance awayfrom a user's skin surface. As just one other example, only ambientlight may be exposed to each light-sensing component (e.g., no light maybe generated by any HWD-internal components) and the more ambient lightdetected may be correlated with a greater distance between the lightsensing component and the user's body (e.g., to detect different userhead shapes and/or different interface fits between the HWD and theuser's head). Any suitable monitoring may be carried out to monitor theproximity and/or contact of one, some, or each light-sensing component,such as any suitable technique, including, but not limited to, providingan IR signal and loop back, or enabling only ambient (non-HWD generatedlight) to be detected by a light-sensing component, or using force orpressure sensors, and/or the like to identify or rank the most reliablelight-sensing components with respect to functional proximity to a user,where each light-sensing component may be weighted or ranked or scoredfor functional proximity and light sensor data from only one or some orall the light-sensing components may be used based on the weighting orranking (e.g., only use data from top ranked (e.g., sensors whose scorepasses a functionally viable proximity threshold), or weight the datafrom each sensor based on proximity functionality rank, etc.). Based onsuch determined functional proximity viability, sensor data from a firstset of one or more light sensor components may be used to determine afirst gesture while a second set of one or more light sensor componentsdifferent from the first set may be used to determine a second gesturedifferent from the first gesture (e.g., using different gesture models).

At operation 906 of process 900, a signal quality for the output of one,some, or each light-sensing component of the HWD may be determined. Forexample, any suitable noise analysis and/or band pass filter may be usedto determine if at least an appropriate amount of signal remains and/orpeak exists for the particular channel (e.g., to make sure a signal hasmost of its energy within an appropriate band of energy and/or harmonics(e.g., to make sure it is not white noise)). Various tests and/orcalibration techniques may be applied to improve the signal(s) from oneor more particular channels during one or more iterations of operation906, such as by adjusting the output strength of at least onelight-emitting component associated with the channel, periodicallymonitoring each channel to determine if the channel's light-sensingcomponent(s) have been saturated by ambient light (e.g., the sun) (e.g.,whereby ambient light and/or HWD light-emitting components may be usedfor different sides of the HWD), turning off each HWD light-emittingcomponent for a channel and taking a dark sample and then taking a lightsample with one or each HWD light-emitting component turned on (e.g., todetermine what dynamic range the channel may be in and/or to determineif a current or strength of any component(s) of the channel ought to beadjusted for obtaining useful data from the channel), and/or the like.Different signal qualities may be required for different applications orfor determinations of different gestures. Based on such determinedsignal quality, sensor data from a first set of one or more light sensorcomponents may be used to determine a first gesture while a second setof one or more light sensor components different from the first set maybe used to determine a second gesture different from the first gesture(e.g., using different gesture models).

At operation 908 of process 900, motion sensor data and/or any othersuitable additional-sensor data of the HWD may be determined (e.g.,using at least one additional-sensing component of a light-sensingassembly of the HWD or otherwise). For example, at least one sensor(e.g., an accelerometer) of the HWD may be operative to provide dataindicative of the motion of the HWD. Any suitable processing of theHWD's system may be used to determine whether or not the associatedlight-sensing component(s) are functionally proximate the functionalsurface in order to determine whether the associated light sensor datamay be relied upon (e.g., at all or with a particular weight) or whetherthe associated light sensor data ought to be disregarded (e.g., atoperation 910). For example, a light-sensing component held against auser's skin surface with a particular force or pressure or within aparticular range thereof may provide more reliable light sensor datathan a light-sensing component determined to be held at a distance awayfrom a user's skin surface. Any suitable processing of system 1 may beused to identify the type of motion being experienced by the HWD andused to determine whether or not to ignore or selectively filter lightsensor data detected during a particular type of motion (e.g., atoperation 910). For example, if the HWD is determined to be moving in acar (e.g., on a bumpy road) that motion may be determined to result inuntrustworthy light sensor data and any data detected during that motionmay be disregarded. Any other suitable non-light sensing data may alsobe determined at operation 908 from one, some, or each additional sensoravailable to the HWD device (e.g., any suitable sound sensor,temperature sensor, etc.).

At operation 910 of process 900, any light sensor data obtained atoperation 902 may be filtered (e.g., removed, weighted, conditioned,combined, averaged, etc.) using one, some, or each determination made atone, some, or each of operations 904, 906, and 908 (e.g., certain lightsensor data may be ignored or weighted based on a determined functionalproximity of its light-sensing component, and/or based on a determinedsignal quality of the data and/or of its light-sensing component, and/orbased on a determined motion of the HWD, and/or based on any othersensed data from any other sensing component of the HWD's system). Anysuitable techniques may be used to provide any suitable filtering forimproving any suitable gesture determination, including, but not limitedto, combining different channels and/or averaging them into a newlydefined channel (e.g., dual sensors (e.g., like sensors positioned onopposite sides of the head of a user) may be averaged for noise removal(e.g., constructive interference may exist for certain biometriccharacteristic gestures (e.g., as detectable heart rate may be the sameon each side of the head) but noise may be destructive interference soan averaging may work)), weighting and/or scoring certain channels basedon functional proximity and/or movement and/or signal quality and/or thelike and then ignoring or devaluing channels based on that weighting orscoring, and/or providing different band pass filters and providingdifferent signals for different type of gesture determinations (e.g., afirst band pass filter may be used to provide signals to be used fordetermining heart rate gesture and a second different band pass filtermay be used to provide signals to be used for determining a breathingrate gesture and a third different band pass filter may be used toprovide signals to be used for determining a soft vocalization gesture).Additionally or alternatively, filtering may include adjusting afunctionality of one or more light-sensing components and/orlight-emitting components and/or additional-sensing components of theHWD to improve signal quality for certain gesture determination, such asby increasing a sampling frequency of a light-sensing component and/orincreasing the brightness of a light-emitting component if a highersignal-to-noise ratio (SNR) is sought (e.g., for determining themovement or velocity of a user's blood stream (e.g., for providing atachogram)).

At operation 912 of process 900, at least one gesture may be determinedusing one, some, or each channel of light sensor data as may be obtainedand filtered by operations 902-910 and using any of the determinationsof operations 904-908. For example, at operation 912, any suitablegesture model may use any suitable channels of light sensor data, whichmay be filtered in any suitable manner, alone or in combination with anysuitable other sensor data for determining (e.g., estimating) one ormore particular gestures (e.g., as described with respect to operation810 of process 800). Therefore, different channels of data may beselected for use or not used based on various ones or more of operations902-910 for determination of different gestures. Gestures may be learnedthrough any suitable learning process (e.g., of process 800), where thesystem (e.g., with an HWD user) may train the system to learn severaldifferent gestures. Feedback may be provided to the user to inform theuser if two or more gestures are similar to one another (e.g., based onconfidence metrics of one or more models). Different gestures can becombined (e.g., a chewing gesture or a particular heart rate gesture maybe combined with interpreted internal voicing (e.g., inaudible,intentional internal voicings) or other sound cues by the user.Biomarkers for different experiences or gestures may be monitored andmay yield and event-reaction (e.g., an anticipation during a sportingevent may lead to a gasp and/or an adrenaline rise and/or an increase inheart rate). Then, at operation 914 of process 900, any gesture(s)determined at operation 912 may then be used (e.g., alone or incombination with any suitable conditions or thresholds or the like) toprovide one or more outputs that may be used to control thefunctionality of the system in any suitable manner(s).

It is understood that the operations shown in process 900 of FIG. 9 areonly illustrative and that existing operations may be modified oromitted, additional operations may be added, and the order of certainoperations may be altered.

FIG. 10 is a flowchart of an illustrative process 1000 for dynamicallyselecting light sensor data channels for potential use in gesturedetection based on ambient light exposure. At operation 1002 of process1000, at the start of a new period (e.g., of any suitable length oftime), for a particular light-sensing assembly of a head-wearabledevice, it may be determined if at least one light-sensing component ofthat light-sensing assembly is at least X % full scale when exposed toonly ambient light. For example, with respect to light-sensing assembly114 b of HWD 100, it may be determined if at least one of light-sensingcomponent (e.g., of light-sensing components 124, 134, and 144 of justside 101 ih of assembly 114 b or of light-sensing components 124′, 134′,and 144′ of just side 101 eh of assembly 114 b or of light-sensingcomponents 124, 124′, 134, 134′, 144, and 144′ of the entirety ofassembly 114 b) is at least X % full scale when exposed to only ambientlight (e.g., ambient light of source AS) and not to any internallygenerated light of HWD 100 (e.g., light of any light-emitting component154 or 154′ of HWD 100). The value of threshold X may be any suitablethreshold, such as any suitable value between 65% and 85% or a value of75%. If none of the light-sensing components of the particularlight-sensing assembly satisfy the requirement of operation 1002 for thecurrent period (e.g., no light-sensing component of the assembly issaturated or nearly saturated), then process 1000 may proceed tooperation 1004, where light sensor data for each channel of light sensordata from each light-sensing component of the particular light-sensingassembly may be selected for potential use (e.g., in determining one ormore user gestures (e.g., at operation 810 of process 800 and/or atoperation 912 of process 900)) (e.g., no channel of light sensor data ofthe particular light-sensing assembly may be filtered out and excludedfrom potential use due to ambient light saturation), and then process1000 may return from operation 1004 to operation 1002. However, if atleast one of the light-sensing components of the particularlight-sensing assembly satisfy the requirement of operation 1002 for thecurrent period (e.g., at least one light-sensing component of theassembly is saturated or nearly saturated), then process 1000 mayproceed to operation 1006, where it may be determined if eachlight-sensing component of that light-sensing assembly is at least Y %full scale when exposed to only ambient light. The value of threshold Ymay be any suitable threshold, such as any suitable value between 65%and 85% or a value of 75%, where Y may be greater than, equal to, orless than the value of threshold X. If not all of the light-sensingcomponents of the particular light-sensing assembly satisfy therequirement of operation 1006 (e.g., if at least one light-sensingcomponent of the assembly is not saturated or not nearly saturated),then process 1000 may proceed to operation 1008, where only light sensordata for each channel of light sensor data from each light-sensingcomponent of the particular light-sensing assembly that is at least Z %full scale available after ambient light is taken into account may beselected for potential use (e.g., in determining one or more usergestures (e.g., at operation 810 of process 800 and/or at operation 912of process 900)) (e.g., certain channels of light sensor data of theparticular light-sensing assembly may be filtered out and excluded frompotential use due to ambient light saturation (e.g., a channel may befiltered out if channel is greater than 100%-Z % full scale when exposedonly to ambient light)), and then process 1000 may advance to operation1018. The value of threshold Z may be any suitable threshold, such asany suitable value between 40% and 80% or a value of 50% or 60% or 70%or 80%. For example, where the value of threshold Z may be defined to be80, operation 1008 may select, for potential use, only light sensor datafor each channel of light sensor data from each light-sensing componentof the light-sensing assembly that is less than 20% full scale whenexposed to only ambient light (i.e., light sensor data for each channelof light sensor data from each light-sensing component of thelight-sensing assembly that is at least 80% full scale available).Because operation 1008 may occur when at least one but not alllight-sensing components of the light-sensing assembly is at or near asaturation (e.g., as defined by the value of threshold X at operation1002 and/or by the value of threshold Y at operation 1006), there may bea higher likelihood that other light-sensing components of thelight-sensing assembly, although not currently saturated, mayinstantaneously move into and/or be specifically prone to saturation(e.g., when (i) the user tilts or moves its head and, thus, thehead-wearable device in a specific orientation relative to an ambientlight source, (ii) the head-wearable device moves its position in anyother manner with respect to an ambient light source, etc.), such thatthe value of threshold Z at operation 1008 may be operative to provide aconservative constraint on the type of light-sensor data that may beselected for potential use during process 1000. However, if eachlight-sensing component of the particular light-sensing assembly satisfythe requirement of operation 1006 (e.g., each light-sensing component ofthe assembly is saturated or nearly saturated), then process 1000 mayproceed to operation 1010, where it may be determined if thelight-sensing assembly is required or at least desired for dual sensing(e.g., differential signaling where light sensor data from differentsides of the user's head may be utilized for better gesturedetermination). If it is determined at operation 1010 that thelight-sensing assembly is not to be used for dual sensing, then process1000 may proceed to operation 1012, where no light sensor data for anychannel of light sensor data from any light-sensing component of theparticular light-sensing assembly may be selected for potential use(e.g., in determining one or more user gestures (e.g., at operation 810of process 800 and/or at operation 912 of process 900)) (e.g., allchannels of light sensor data of the particular light-sensing assemblymay be filtered out and excluded from potential use due to ambient lightsaturation), and then process 1000 may return from operation 1012 tooperation 1018. However, if it is determined at operation 1010 that thelight-sensing assembly is to be used for dual sensing, then process 1000may proceed to operation 1014, where a particular channel of thelight-sensing assembly that is exhibiting the largest full scaleavailability and/or the lowest variability over a limited time window(e.g., a sub-period length of time of the current period) may beidentified. Then, process 1000 may proceed to operation 1014, where onlylight sensor data for the identified channel of the particularlight-sensing assembly may be selected for potential use (e.g., indetermining one or more user gestures (e.g., at operation 810 of process800 and/or at operation 912 of process 900)) (e.g., all but a particularidentified channel of the particular light-sensing assembly may befiltered out and excluded from potential use due to ambient lightsaturation), and then process 1000 may advance to operation 1018. Atoperation 1018, for any light-sensing component of any channel of thelight-sensing assembly that has not been selected for potential use forthe current period (e.g., at one of operations 1008, 1012, and 1016), itmay be determined whether that light-sensing component is at least W %full-scale when exposed to only ambient light. The value of threshold Wmay be any suitable threshold, such as any suitable value between 65%and 85% or a value of 75%, which may be greater than or less than orequal to value X and/or greater than or less than or equal to value Y.If at least one of the non-selected light-sensing components of theparticular light-sensing assembly satisfy the requirement of operation1018, then process 1000 may proceed to operation 1020, where it may bedetermined if the current period has ended, and, if so, process 1000 mayreturn to operation 1002, otherwise process 1000 may return to operation1006. However, if none of the non-selected light-sensing components ofthe particular light-sensing assembly satisfy the requirement ofoperation 1018, then process 1000 may proceed to operation 1022, whereit may be determined if the current period has ended, and, if so,process 1000 may return to operation 1002, otherwise process 1000 mayreturn to operation 1018. Process 1000 may be carried out after anysuitable gain options have been exhausted (e.g., no more gain optionsare available for improving the dynamic range of a light-sensingcomponent without saturating that light-sensing component). Saturationdetected by process 1000 if for ambient light saturation and not due toany light emitted by the HWD device itself, in which case a magnitude ofone or more light-emitting components of the HWD may be reduced to avoidsuch saturation (or, if not possible, a non-human surface would beassumed for reflecting the light causing such saturation and may beignored altogether). Similar determinations to those made at one or moreof operations 1002, 1006, 1008, 1014, and 1018 with respect tosaturation and/or full scale availability may be Made with respect tosignal quality (e.g., as an alternative to or in addition to channelselection for high ambient light situations, but for high signal qualitysituations), which can be based on any suitable metric(s), such asstandard deviation (e.g., to avoid channels with random impulses due tocontact modulation), correlation (e.g., with other channels), phasedifference (e.g., with other channels), and/or the like. In someembodiments, where sufficient ambient light is detected but saturationis sufficiently avoided, such ambient light may be used as the lightsource operative to enable one or more channels of light sensor data(e.g., without relying on one or more light-emitting components of theHWD). For example, at least for light-sensing components 124′, 134′, and144′ of ear side 101 eh of assembly 114 b that may be more susceptibleto ambient light (e.g., through a user's ear) as opposed to skull side101 ih of assembly 114 b, those light-sensing components may beoperative to detect ambient light rather than light from light-emittingcomponent 154′ (e.g., during a low power mode where poweringlight-emitting component 154′ may not be desired and/or when significantambient light may be detected by those light-sensing components).Therefore, a passive transmission mode may be used when ambient light issufficiently detected by the HWD via a portion of the user (e.g.,modulated by physiological changes (e.g., blood due to motion) withinthat user portion and/or due to contact and/or pressure variabilitybetween that user portion and the HWD during any suitable usergestures). Alternatively, a reflectance mode may be used whenHWD-generated light is launched from the HWD into the user's body and isscattered within and is reflected back from the user's body to the HWDfor detection, where such scattered and reflected light may be modulatedby physiological changes (e.g., blood due to motion) within that userbody portion and/or due to contact and/or pressure variability betweenthat user body portion and the HWD during any suitable user gestures.Any one or more of light-transmissive elements 125, 125′, 135, 135′,145, and 145′ at any suitable light-sensing component of the HWD may beprovided with any suitable directional preference component that may beoperative to enable the light-sensing component to avoid a direct lightsource (e.g., sunlight, above lit office lighting, etc.) in order toavoid saturation (e.g., by preferentially collecting light incident tothe light-sensing component and not at any other angles).

It is understood that the operations shown in process 1000 of FIG. 10are only illustrative and that existing operations may be modified oromitted, additional operations may be added, and the order of certainoperations may be altered.

Further, although examples of the disclosure may be described hereinprimarily in terms of devices with multiple assemblies and/or multipleof light-sensing components (e.g., multiple photodiodes), it should beunderstood that examples of the disclosure are not so limited, butinclude devices with only a single sensor assembly and/or a singlelight-sensing component (e.g., a single photodiode). A channel of sensordata can correspond to each unique light sensor/emitter pair, whetherthere is one or multiple sensors, one or multiple emitters, and/or thelike.

HWD 100 may include any suitable number of light-sensing assemblies,each with any suitable number of light-sensing components, arranged inany suitable manner, such as strategically placed for sensing one ormore suitable types of physiological signals or movements of a user withrespect to HWD 100 (e.g., movement of a user's skin with respect to HWD100 (e.g., during a detectable chewing or vocalization or facial orother suitable gesture (e.g., when an HWD is put on or removed from auser's head)) and/or movement of a user's blood with respect to HWD 100(e.g., during a detectable change in a heart rate or breathing rate orother biometric characteristic gesture)). Dual placement oflight-sensing assemblies, one on each side of a user's bead when the HWDis properly worn, may enable sensing of highly differential signals forimproving the accuracy of one or more types of gesture determination.Differential signaling may take advantage of multiple locations ofsensors of the HWD with respect to a user's body (e.g., opposite sidesof a user's head, above each ear, above each temple, etc.), which mayallow the HWD to capture a lot more information (e.g., more reliablevital signs) than if one sensor positioned at one location and/orinformation with less noise (e.g., less noise than if a sensor is onlypositioned on a user's wrist or at one of the user's ears, which may besusceptible to a lot of motion artifacts (e.g., wind/etc.), whichvarious HWDs of this disclosure may avoid). Moreover, the vasculature isdifferent and simpler in a user's head than in a user's wrist, which mayprovide significant advantages to the efficiency and effectiveness ofthe HWDs of this disclosure. An HWD may be biased against a user's headand may provide a stable and/or reliable interface with a user ascompared to a sensing device that may be worn on a user's wrist or ear,which may be more exposed to wind, air, ambient light, internalmovements, ligaments, and/or the like than a user's head may be. Theplacement of and/or distance between sensors that may be specificallyafforded by an HWD may be used to enhance the signal to noise ratio. Forexample, a contour of an HWD may be provided to exert a pressure on auser's head for maintaining an interface between the user and a sensingassembly. A distance between a light-sensing component and alight-emitting component may be optimized to the geometry of a surfaceof a head, such as based on bone or structure of a skull. Thewavelengths of light emitted by one or more light-emitting componentsmay be chosen depending on the location of assumed contact of the headand/or level of motion. For example, emitted light at IR wavelength(s)may be preferred in regions of the HWD that may make better contact withthe user and/or that have less motion with respect to the user duringuse, while emitted light at green wavelength(s) may be preferred inregions of the HWD that may regularly move more with respect to the useror otherwise during use. In regions of the HWD where it may be assumedthat there will be contact modulation between the HWD and the user, asource-to-detector spacing (e.g., between light-transmissive element 155of LE1 154 and light-transmissive element 125 of PD1 124) may be chosento yield a proximity curve that may be relatively flat (e.g., in a rangeof contact to 4.0 millimeters). This may reduce the impact of motionartifacts. A fresnel lens may be used to collimate or steer a beam toachieve a desired proximity curve. Multiple sets of light-sensingcomponents and light-emitting components may be provided along thelength of one or each arms of a glasses-type or other suitable type ofHWD for accommodating various user head geometries while maintaining aneffective sensor/user interface. Additionally, an HWD may be configuredwith one or more sensors to be positioned between a user's eyebrows(e.g., at a bridge of the nose) and/or behind the user's ears, which maybe helpful for detecting any suitable gestures related to any electricalactivity of the brain (e.g., as appropriate for enabling effectiveelectroencephalography (EEG) (e.g., for predicting or diagnosing orotherwise utilizing the detection of potential user epilepsy, sleepdisorder(s), encephalopathy, tumor, stroke, and/or the like)). Ambientlight may be used to detect any suitable gestures. The HWD need not justlook at an internally-emitted light channel (e.g., light channel)subtracting an ambient-emitted light channel (e.g., dark channel), butcould be operative to use the dark channel itself as a signature forinput to a template bank for comparison or to any suitable gesturemodel. Additionally or alternatively, the HWD may be operative such thatany suitable light-sensing component may be enabled to only detectambient light, while such detection may be used to adjust a brightnessof a display output component or any other functionality of the HWDsystem.

For mitigating certain possible ambient light issues (e.g., degradationof HWD effectiveness due to ambient light, at least in embodiments thatmay leverage IR wavelength(s), an IR-transparent (but opaque to visiblelight) ink may be provided over one, some, or each one of thelight-sensing components (e.g., light-transmissive element 125 of PD1124 may be provided with any suitable IR-transparent ink or othersuitable material). In some embodiments, an IR-transparent but visiblelight opaque ink may be provided along an exterior surface of a majorityor the entirety of the HWD (e.g., for cosmetic purposes). However, ifsome estimate of ambient light may be useful (e.g., to estimate contactto skin, etc.), some of the light-sensing components can be covered bythe ink, while other light-sensing components may not be covered by theink, where the uncovered light-sensing components may be used at leastfor ambient light detection while the covered light-sensing componentsmay be used for IR detection. If a mix of visible (e.g., Red and/orGreen) light and IR light are to be used, then the IR-transparent inkmay be used to cover IR LED die (e.g., cosmetics) and light-sensingcomponents that may be used primarily for IR light collection (e.g., dueto their proximity with the IR LED die, etc.). During certain activitiesor usage cases, the number of sampled optical channels can be reducedfor power savings. The usage cases can include determining “still” fromany gesture (e.g., where very limited accuracy on which gesture wasperformed may be acceptable) or a very small subset of the complete listof gestures. Besides monitoring the ambient light mean level, the HWDsystem may be configured to monitor the ambient light noise (i.e., darkchannel noise) to make sure that it is sufficiently small compared tothe signatures being searched for in the gesture detection. If theambient light noise is too high in some of the receive channels, thenthis may also be cause to drop these channels from the decision makingprocess (e.g., this could essentially be baked into the signal qualitycheck for each optical channel (e.g., before allowing that opticalchannel to be counted towards gesture detection)).

The sensing capabilities of such an HWD may enable the ability tomeasure various suitable biometric characteristic gestures, such asheart rate (HR) and heart rate variability (HRV), where HRV maytypically be associated with stress. Vital sign detection may be enabledby the various arrangements and uses of light-sensing and light-emittingcomponents along various portions of various HWDs of this disclosure.For example, an average biometric characteristic (e.g., HR) value over8-10 second window (e.g., using fast Fourier transform (FFT)) may bedetermined to report that biometric characteristic every 5 or 10seconds. Alternatively, a beat-to-beat determination of heart rate maybe determined to provide a record (e.g., a tachogram) of the movementand/or velocity of the bloodstream (e.g., as may be made by atachometer), where a quick change in heart rate beat to beat (e.g., agasp) and/or based on any other suitable gestures (e.g., based on whatsituation a user may be in), may enable significant advantages. Forexample, an HWD may be operative to turn on any suitable process (e.g.,a tachogram algorithm) for certain situations (e.g., to increase currentto one or more light-emitting components (e.g., to increase SNR) and/orto increase sampling frequency of one or more light-sensing components),such as when user-initiated or when a gasp gesture is detected (e.g.,opportunistically (monitoring accelerometer or IR channel, but noticegesture occurred that may trigger the new mode (e.g., for a minute,etc.))) or when application activated (e.g., during a certain game modewhere it may be useful to specifically measure a user's beat-to-beatresponse (e.g., heart rate and/or heart rate turbulence (HRT) and/orbreathing rate) to something that happens during that game mode).

The sensing capabilities of such an HWD may enable the ability tomeasure various suitable intentional user gestures, including, but notlimited to, a stationary gesture, a detectable chewing gesture, anopening mouth gesture, a closing mouth gesture, a gasping gesture, anyvocalization gesture (e.g., any soft vocalization gesture (e.g., a usersaying aloud or internally or under breath, “yeah” or “do that” or “mmmhmmm” or “uh huh” or “nuh uh” or the like)), any suitable brain function(e.g., using functional near-infrared spectroscopy (fNIR or fNIRS)) fordetecting whether a user is reading or alert or glazed or anxious or thelike (e.g., using differential signaling (e.g., with one or more lightchannels between 700 and 900 nanometers or the like)), any facialmovement gesture, any gesture for removal of the HWD from the user'shead, any gesture for positioning of the HWD on the user's head, and/orthe like. Synergy between light sensing and other data sensing methods(e.g., sound and/or pressure sensing) may yield accurate bio markers ofa user's gestures (e.g., any combination of detected light-sensingsignals and detected audio information and/or detected pressureinformation or force information and/or detected temperature informationand/or detected location information and/or the like may be used to moreeffective and/or efficiently attempt to determine or estimate one ormore user gestures. While a user is gaming (e.g., playing a video game)while wearing an HWD, the HWD may measure HR & HR fluctuations tomonitor the user (e.g., for them to see live feedback of their vitalsduring highly intense moments of game play), which may enable a moreimmersive experience for the user. Even during everyday tasks of a userwearing an HWD, the HWD may detect various motions that may enable ahands-free method to perform various tasks (e.g., automatically record auser's environment in response to detection of any suitable usergesture) and/or enable additional accessibility options for users withlimited mobility. An HWD may provide experience enhancement andsimplification, such as by monitoring any suitable gesture cues from auser for event driven actions (e.g., capture the moment when anysuitable biomarker suggests it is an important event, and record thedate/time/location (e.g., while a user is wearing the HWD at sportingevents, gaming, special moments, all with limited need for userinteraction (e.g., no need to pull out your phone, for taking picture,etc.))). For example, when a user wearing an HWD eats breakfast in themorning, the HWD may be operative to detect the user's chewing tomonitor how frequently the user is eating and the duration thereofand/or detect a chewing intensity to gain insight into the types of foodthe user may be eating, which may be used by the HWD system to recommendhealthier eating frequency and/or foods. As another example, when a userwearing an HWD goes to work and becomes stressed out, the HWD may beoperative to detect increases in the user's heart rate and/orrespiration rate and/or changes in any other suitable user gestures(e.g., teeth grinding, vocalizations of “ugh”, etc.), which may be usedby the HWD system to recommend that the user meditate and/or rest, whichmay be confirmed by the HWD detecting that the user has closed its eyes.As another example, when a user wearing an HWD gets home from work andplays a video game, the HWD may be operative to detect facial and oculargestures (e.g., squinting and/or blinking) to augment any other videogame input controller functionality (e.g., instruct the user's gamingcharacter to jump every time a squint gesture is detected) and/or todetect increases in the user's heart rate and/or respiration rate and/orchanges in any other suitable user gestures to adjust a gamingexperience (e.g., to present detected user biometric characteristics tothe gaming user and/or to adjust a difficulty level of the gameresponsive to such HWD detections). As another example, when a userwearing an HWD goes to a bar to watch a sporting event with friends, theHWD may be operative to detect facial and ocular gestures (e.g., smilingand other arousal), which may be used to automatically capture aphotograph or video during moments of heightened user excitement (e.g.,to automatically capture an important moment of the user).

Therefore, this disclosure relates to detecting head gestures using anelectronic device, such as a head wearable device held against anysuitable portion(s) of a user's head. The device can have multiplelight-sensing components (e.g., photodiodes), each sensing light at adifferent position on a surface of the device that faces skin of a user(as well as, optionally, one or more surfaces of the device that mayface one or more ambient light sources). Due to this positioning, thesensor data from the light-sensing components can capture movement ofanatomical features in the tissue of the user during a head gesture.Further, different light-emitting components on the device can emitlight at different wavelengths (e.g., infrared light, green light,etc.), which may penetrate to different depths in the tissue of theuser's head before reflecting back to the light-sensing components onthe device. Accordingly, sensor data from the light-sensing componentscan capture expansion and contraction in the tissue of the user's headduring a head gesture. Examples of the disclosure may detect headgestures by recognizing patterns in sensor data that may becharacteristic of each head gesture, as the tissue expands and contractsand anatomical features in the tissue move during the gesture. Althoughexamples of the disclosure may be described herein primarily in terms ofwearable devices strapped to a head and head gestures, particularlybiometric characteristic gestures and intentional and/or involuntaryfacial and brain gestures, it should be understood that examples of thedisclosure are not so limited, but include wearable devices attached toother body parts, such as a neck and/or upper arms or legs, and gesturesthat can result therefrom. FIGS. 2-21 may illustrate exemplary HWDs witha plurality of sensors in accordance with examples of the disclosure.Each HWD can include a plurality of light-sensing components and anysuitable number of light-emitting components. When an HWD is in use, thelight-sensing components and the light-emitting component(s) may facethe tissue of a user's head. Each light-sensing component can senselight at a different position on a surface of the device that faces thetissue of the user's head. Due to this positioning, the sensor data fromthe light-sensing components can capture movement of anatomical featuresin the tissue of the user during one or more head gestures. Differentlight-emitting components on an HWD can emit light at differentwavelengths (e.g., infrared light, green light, etc.), which maypenetrate to different depths in the tissue of the user's head beforereflecting back to the light-sensing components on the device.Accordingly, sensor data from the light-sensing components can captureexpansion and contraction in the tissue of the user's head during a headgesture. In some examples, sensor data (e.g., the first, second, orthird light sensor channel data) can be further processed beforeextracting signal characteristics. For example, a band pass filter maybe applied to sensor data to filter out heart rate frequencies from thesensor data. As light sensor data may vary according to the periodicmotion of blood through human tissue, it may be beneficial to filter outthese frequencies to better isolate the contribution of head gesturemotion to the signal characteristics.

This disclosure relates to a data processing system (e.g., system 1) forextracting a desired vital signal of a user wearing an HWD of thesystem, where the vital signal may contain a physiological informationcomponent pertaining to a subject of interest, from photoplethysmographydata. It also relates to a photoplethysmography system, to a dataprocessing method for extracting a desired vital signal, which maycontain a physiological information component pertaining to a subject ofinterest, from photoplethysmography data, and to a computer program.Information about cardiovascular status, such as blood oxygensaturation, heart and respiratory rates, and/or the like can beunobtrusively acquired by photoplethysmography (PPG) using sensors, suchas light-sensing components on the HWD and/or any suitableadditional-sensing components (e.g., sound sensing components and/orforce sensing components and/or the like). PPG may be used for anestimation of cardiovascular parameters. This technique has beenpreferred over other techniques such as a chest belt forelectrocardiography (ECG) or an electronic stethoscope because thelatter two are often considered as a reduction in comfort and usability.However, a motion of the subject of interest (e.g., a wearer of an HWDof this disclosure) during a PPG measurement may generate motionartifacts in measured PPG signals, which may lead to erroneousinterpretation and degrade the accuracy and reliability of estimation ofcardiovascular parameters if such artifacts are not reduced or fullyremoved by the system. PPG data is data may be obtained by a PPGmeasurement before it is provided to a data processing device. The PPGdata may for instance be provided in the form of sensor data generatedby one or more light-sensing components on the device (e.g., one or morephotodiodes or cameras), and may indicate a detected amount of lightemitted by one or more light-emitting components (e.g., one or more LEDsor laser diodes) and reflected from or, depending on the measurementsetup, transmitted through a sensed region of a subject of interest as afunction of time. The sensed region may be a region of the skin of thesubject of interest's head when wearing the HWD.

Moreover, one, some, or all of the processes described with respect toFIGS. 1-10 may each be implemented by software, but may also beimplemented in hardware, firmware, or any combination of software,hardware, and firmware. They each may also be embodied as machine- orcomputer-readable code recorded on a machine- or computer-readablemedium. The computer-readable medium may be any data storage device thatcan store data or instructions which can thereafter be read by acomputer system. Examples of such a non-transitory computer-readablemedium (e.g., memory assembly 104 of FIG. 1) may include, but are notlimited to, read-only memory, random-access memory, flash memory,CD-ROMs, DVDs, magnetic tape, removable memory cards, optical datastorage devices, and the like. The computer-readable medium can also bedistributed over network-coupled computer systems so that thecomputer-readable code is stored and executed in a distributed fashion.For example, the computer-readable medium may be communicated from oneelectronic device to another electronic device using any suitablecommunications protocol (e.g., the computer-readable medium may becommunicated to electronic device 100 via any suitable communicationsassembly 106 (e.g., as at least a portion of application 103)). Such atransitory computer-readable medium may embody computer-readable code,instructions, data structures, program modules, or other data in amodulated data signal, such as a carrier wave or other transportmechanism, and may include any information delivery media. A modulateddata signal may be a signal that has one or more of its characteristicsset or changed in such a manner as to encode information in the signal.

It is to be understood that any or each module of sensor managementsystem 301 may be provided as a software construct, firmware construct,one or more hardware components, or a combination thereof. For example,any or each module of sensor management system 301 may be described inthe general context of computer-executable instructions, such as programmodules, that may be executed by one or more computers or other devices.Generally, a program module may include one or more routines, programs,objects, components, and/or data structures that may perform one or moreparticular tasks or that may implement one or more particular abstractdata types. It is also to be understood that the number, configuration,functionality, and interconnection of the modules of sensor managementsystem 301 are only illustrative, and that the number, configuration,functionality, and interconnection of existing modules may be modifiedor omitted, additional modules may be added, and the interconnection ofcertain modules may be altered.

At least a portion of one or more of the modules of sensor managementsystem 301 may be stored in or otherwise accessible to device 100 in anysuitable manner (e.g., in memory assembly 104 of device 100 (e.g., as atleast a portion of application 103)). Any or each module of sensormanagement system 301 may be implemented using any suitable technologies(e.g., as one or more integrated circuit devices), and different modulesmay or may not be identical in structure, capabilities, and operation.Any or all of the modules or other components of sensor managementsystem 301 may be mounted on an expansion card, mounted directly on asystem motherboard, or integrated into a system chipset component (e.g.,into a “north bridge” chip).

Any or each module of sensor management system 301 may be a dedicatedsystem implemented using one or more expansion cards adapted for variousbus standards. For example, all of the modules may be mounted ondifferent interconnected expansion cards or all of the modules may bemounted on one expansion card. With respect to sensor management system301, by way of example only, the modules of sensor management system 301may interface with a motherboard or processor assembly 102 of device 100through an expansion slot (e.g., a peripheral component interconnect(“PCI”) slot or a PCI express slot). Alternatively, sensor managementsystem 301 need not be removable but may include one or more dedicatedmodules that may include memory (e.g., RAM) dedicated to the utilizationof the module. In other embodiments, sensor management system 301 may beat least partially integrated into device 100. For example, a module ofsensor management system 301 may utilize a portion of device memoryassembly 104 of device 100. Any or each module of sensor managementsystem 301 may include its own processing circuitry and/or memory.Alternatively, any or each module of sensor management system 301 mayshare processing circuitry and/or memory with any other module of sensormanagement system 301 and/or processor assembly 102 and/or memoryassembly 104 of device 100.

As described above, one aspect of the present technology is thegathering and use of data available from various sources to determineone or more head gestures of a user (e.g., a wearer of an HWD). Thepresent disclosure contemplates that in some instances, this gathereddata may include personal information data that uniquely identifies orcan be used to contact or locate a specific person. Such personalinformation data can include demographic data, location-based data,telephone numbers, email addresses, social network identifiers, homeaddresses, office addresses, data or records relating to a user's healthor level of fitness (e.g., vital signs measurements, medicationinformation, exercise information, etc.) and/or mindfulness, date ofbirth, or any other identifying or personal information.

The present disclosure recognizes that the use of such personalinformation data, in the present technology, can be used to the benefitof users. For example, the personal information data can be used toimprove the determination of sensor states of a user. Further, otheruses for personal information data that benefit the user are alsocontemplated by the present disclosure. For instance, health and fitnessdata may be used to provide insights into a user's general wellness, ormay be used as positive feedback to individuals using technology topursue wellness goals.

The present disclosure contemplates that the entities responsible forthe collection, analysis, disclosure, transfer, storage, or other use ofsuch personal information data will comply with well-established privacypolicies and/or privacy practices. In particular, such entities shouldimplement and consistently use privacy policies and practices that aregenerally recognized as meeting or exceeding industry or governmentalrequirements for maintaining personal information data private andsecure. Such policies should be easily accessible by users, and shouldbe updated as the collection and/or use of data changes. Personalinformation from users should be collected for legitimate and reasonableuses of the entity and not shared or sold outside of those legitimateuses. Further, such collection/sharing should occur after receiving theinformed consent of the users. Additionally, such entities shouldconsider taking any needed steps for safeguarding and securing access tosuch personal information data and ensuring that others with access tothe personal information data adhere to their privacy policies andprocedures. Further, such entities can subject themselves to evaluationby third parties to certify their adherence to widely accepted privacypolicies and practices. In addition, policies and practices should beadapted for the particular types of personal information data beingcollected and/or accessed and adapted to applicable laws and standards,including jurisdiction-specific considerations. For instance, in theUnited States, collection of or access to certain health data may begoverned by federal and/or state laws, such as the Health InsurancePortability and Accountability Act (“HIPAA”); whereas health data inother countries may be subject to other regulations and policies andshould be handled accordingly. Hence different privacy practices shouldbe maintained for different personal data types in each country.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data. That is, the present disclosure contemplatesthat hardware and/or software elements can be provided to prevent orblock access to such personal information data. For example, in the caseof location detection services, the present technology can be configuredto allow users to select to “opt in” or “opt out” of participation inthe collection of personal information data during registration forservices or anytime thereafter. In addition to providing “opt in” or“opt out” options, the present disclosure contemplates providingnotifications relating to the access or use of personal information. Forinstance, a user may be notified upon downloading an app that theirpersonal information data will be accessed and then reminded again justbefore personal information data is accessed by the app.

Moreover, it is the intent of the present disclosure that personalinformation data should be managed and handled in a way to minimizerisks of unintentional or unauthorized access or use. Risk can beminimized by limiting the collection of data and deleting data once itis no longer needed. In addition, and when applicable, including incertain health related applications, data de-identification can be usedto protect a user's privacy. De-identification may be facilitated, whenappropriate, by removing specific identifiers (e.g., date of birth,etc.), controlling the amount or specificity of data stored (e.g.,collecting location data a city level rather than at an address level),controlling how data is stored (e.g., aggregating data across users),and/or other methods.

Therefore, although the present disclosure broadly covers use ofpersonal information data to implement one or more various disclosedembodiments, the present disclosure also contemplates that the variousembodiments can also be implemented without the need for accessing suchpersonal information data. That is, the various embodiments of thepresent technology are not rendered inoperable due to the lack of all ora portion of such personal information data. For example, thedetermination of sensor states of a user of an electronic device can bemade based on non-personal information data or a bare minimum amount ofpersonal information, such as the content being requested by the deviceassociated with a user, other non-personal information available to thedevice, or publicly available information.

While there have been described systems, methods, and computer-readablemedia for monitoring a user of a head-wearable electronic device withmultiple light-sensing assemblies, it is to be understood that manychanges may be made therein without departing from the spirit and scopeof the subject matter described herein in any way. Insubstantial changesfrom the claimed subject matter as viewed by a person with ordinaryskill in the art, now known or later devised, are expressly contemplatedas being equivalently within the scope of the claims. Therefore, obvioussubstitutions now or later known to one with ordinary skill in the artare defined to be within the scope of the defined elements. It is alsoto be understood that various directional and orientational terms, suchas “up” and “down,” “front” and “back,” “top” and “bottom” and “side,”“above” and “below,” “length” and “width” and “thickness” and “diameter”and “cross-section” and “longitudinal,” “X-” and “Y-” and “Z-,” and thelike, may be used herein only for convenience, and that no fixed orabsolute directional or orientational limitations are intended by theuse of these terms. For example, the components of an HWD can have anydesired orientation. If reoriented, different directional ororientational terms may need to be used in their description, but thatwill not alter their fundamental nature as within the scope and spiritof the subject matter described herein in any way.

Therefore, those skilled in the art will appreciate that the concepts ofthe disclosure can be practiced by other than the described embodiments,which are presented for purposes of illustration rather than oflimitation.

What is claimed is:
 1. An electronic device comprising: a head-wearablehousing structure comprising: an eye frame; a right temple frameextending from the eye frame; and a left temple frame extending from theeye frame, wherein, when the head-wearable housing structure is worn ona head of a user, the head-wearable housing structure is configured suchthat: the eye frame is positioned in front of at least one eye of theuser's head; the right temple frame is held against a right surface ofthe user's head; and the left temple frame is held against a leftsurface of the user's head; a right light-sensing assembly supported bythe right temple frame, comprising: a right light-emitting componentoperative to emit light into the right surface of the user's head whenthe head-wearable housing structure is worn on the head of the user; anda right light-sensing component held against the right surface of theuser's head and operative to sense right light emitted from the rightsurface of the user's head when the head-wearable housing structure isworn on the head of the user, the right light comprising at least aportion of the light emitted into the right surface of the user's headby the right light-emitting component when the head-wearable housingstructure is worn on the head of the user; a left light-sensing assemblysupported by the left temple frame, comprising: a left light-emittingcomponent operative to emit light into the left surface of the user'shead when the head-wearable housing structure is worn on the head of theuser; and a left light-sensing component held against the left surfaceof the user's head and operative to sense left light emitted from theleft surface of the user's head when the head-wearable housing structureis worn on the head of the user, the left light comprising at least aportion of the light emitted into the left surface of the user's head bythe left light-emitting component when the head-wearable housingstructure is worn on the head of the user; and a processor operative to:analyze light data indicative of the sensed right light and the sensedleft light; and determine a head gesture of the user based on theanalyzed light data.
 2. The electronic device of claim 1, wherein theprocessor is further operative to automatically adjust a functionalityof the head-wearable electronic device based on the determined headgesture.
 3. The electronic device of claim 1, wherein the determinedhead gesture comprises one of the following: chewing; smiling; frowning;grimacing; gasping; mouth opening; mouth closing; or humming.
 4. Theelectronic device of claim 1, wherein the processor is further operativeto determine a biometric characteristic of the user based on theanalyzed light data, wherein the determined biometric characteristiccomprises one of the following: a particular heart rate; or a particularheart rate variability.
 5. The electronic device of claim 4, wherein:the processor is further operative to automatically record at least oneimage of the user's environment based on at least one of the determinedhead gesture and the determined biometric characteristic.
 6. Theelectronic device of claim 1, wherein the processor is operative todetermine the head gesture of the user by: collecting first sensor datafrom the left and right light-sensing assemblies during a first periodin which the user performs a first head gesture when the head-wearablehousing structure is worn on the head of the user; collecting secondsensor data from the left and right light-sensing assemblies during asecond period in which the user performs a second head gesture when thehead-wearable housing structure is worn on the head of the user;calculating first signal characteristics based on the first sensor data;calculating second signal characteristics based on the second sensordata; assigning some or all of the first signal characteristics to afirst cluster of signal characteristics; assigning some or all of thesecond signal characteristics to a second cluster of signalcharacteristics; collecting the light data as third sensor dataindicative of the sensed right light and the sensed left light from theleft and right light-sensing assemblies during a third period;calculating third signal characteristics based on the third sensor data;determining whether the third signal characteristics belong to the firstcluster, the second cluster, or a third cluster; determining that theuser has performed the first head gesture in accordance with adetermination that the third signal characteristics belong to the firstcluster; determining that the user has performed the second head gesturein accordance with a determination that the third signal characteristicsbelong to the second cluster and determining that the user has notperformed the first head gesture or the second head gesture inaccordance with a determination that the third signal characteristicsbelong to the third cluster.
 7. The electronic device of claim 6,wherein the processor is further operative to determine the head gestureby: comparing the first cluster to the second cluster; determining thereare more of the first signal characteristics assigned to the firstcluster than to the second cluster based on comparing the first clusterto the second cluster; and assigning the first cluster to the first headgesture in accordance with the determination that there are more of thefirst signal characteristics assigned to the first cluster than to thesecond cluster.
 8. The electronic device of claim 7, wherein theprocessor is further operative to determine the head gesture by:determining there are more of the second signal characteristics assignedto the second cluster than to the first cluster based on comparing thefirst cluster to the second cluster; and assigning the second cluster tothe second head gesture in accordance with the determination that thereare more of the second signal characteristics assigned to the secondcluster than to the first cluster.
 9. The electronic device of claim 6,wherein the processor is further operative to determine the head gestureby: generating a first template for the first head gesture; generating asecond template for the second hand gesture; and comparing the thirdsignal characteristics to the first and second templates.
 10. Theelectronic device of claim 9, wherein the processor is further operativeto determine the head gesture by: calculating first mean signalcharacteristics for the first cluster, wherein the first template isgenerated based on the first mean signal characteristics for the firstcluster; and calculating second mean signal characteristics for thesecond cluster, wherein the second template is generated based on thesecond mean signal characteristics for the second cluster.
 11. Theelectronic device of claim 10, wherein the processor is operative tocompare the third signal characteristics to the first and secondtemplates by: calculating a first distance from the third signalcharacteristics to the first template and calculating a second distancefrom the third signal characteristics to the second template;determining that the third signal characteristics belong to the firstcluster in accordance with a determination that the first distance isshorter than the second distance; and determining that the third signalcharacteristics belong to the second cluster in accordance with adetermination that the second distance is shorter than the firstdistance.
 12. The electronic device of claim 9, wherein: the processoris operative to generate the first template for the first head gestureby storing some or all of the first sensor data as the first template;and the processor is operative to generate the second template for thesecond head gesture by storing some or all of the second sensor data asthe second template.
 13. The electronic device of claim 6, wherein theprocessor is operative to calculate the first signal characteristics bycalculating at least one of the following: an amplitude differencebetween a peak and a trough of the first sensor data; a time differencebetween a peak and a trough of the first sensor data; a maximumamplitude of the first sensor data; a period between peaks of the firstsensor data; or a phase of the first sensor data.
 14. The electronicdevice of claim 6, wherein the processor is further operative todetermine the head gesture by filtering heart rate frequencies from thefirst sensor data before calculating the first signal characteristicsbased on the first sensor data.
 15. The electronic device of claim 6,wherein the processor is further operative to determine the head gestureby further collecting the first sensor data from at least one of a forcesensor and an accelerometer during the first period in which the userperforms the first head gesture.
 16. The electronic device of claim 6,wherein the processor is further operative to determine the head gestureby: collecting fourth sensor data from the left and right light-sensingcomponents during a fourth period in which the user does not perform thefirst or second head gesture; calculating fourth signal characteristicsbased on the fourth sensor data; and assigning some or all of the fourthsignal characteristics to the third cluster of signal characteristics.17. The electronic device of claim 16, wherein the processor isoperative to assign the first and second signal characteristics to thefirst and second clusters by using a k-means clustering algorithm. 18.The electronic device of claim 17, wherein: the k-means clusteringalgorithm is also applied to the third signal characteristics; and theprocessor is operative to determine whether the third signalcharacteristics belong to the first cluster, the second cluster, or thethird cluster based on the k-means clustering algorithm.
 19. Theelectronic device of claim 6, wherein the first head gesture comprisesone of the following: chewing; smiling; frowning; grimacing; mouthopening; mouth closing; removing the electronic device from the user'shead; adorning the user's head with the electronic device; humming; oreyebrow raising.
 20. The electronic device of claim 19, wherein thesecond head gesture comprises a different one of the following than thefirst head gesture: chewing; smiling; frowning; grimacing; gasping;mouth opening; mouth closing; removing the electronic device from theuser's head; adorning the user's head with the electronic device;humming; or eyebrow raising.
 21. The electronic device of claim 6,wherein the processor is further operative to determine a biometriccharacteristic of the user based on the analyzed light data, wherein thedetermined biometric characteristic comprises one of the following: aparticular heart rate; or a particular heart rate variability.
 22. Theelectronic device of claim 1, further comprising a head gesture modelcustodian system that is configured to monitor the user when thehead-wearable housing structure is worn on the head of the user by:initially configuring, at the head gesture model custodian system, alearning engine; receiving, at the head gesture model custodian system,sensor category data for at least one sensor category for a traininghead gesture and a score for the training head gesture; training, at thehead gesture model custodian system, the learning engine using thereceived sensor category data and the received score; accessing, at thehead gesture model custodian system, sensor category data for the atleast one sensor category for the head gesture of the user from thelight data indicative of the sensed right light and the sensed leftlight; scoring the head gesture of the user, using the learning engineat the head gesture model custodian system, with the accessed sensorcategory data for the at least one sensor category data for the headgesture of the user; and when the score for the head gesture of the usersatisfies a condition, generating, with the head gesture model custodiansystem, control data associated with the satisfied condition.
 23. Anelectronic device comprising: a head-wearable housing structure,wherein, when the head-wearable housing structure is worn on a head of auser, the head-wearable housing structure is configured such that afirst portion of the head-wearable housing structure is held against afirst surface of the user's head and such that a second portion of thehead-wearable housing structure is held against a second surface of theuser's head; a first light-sensing assembly supported by the firstportion of the head-wearable housing structure, comprising: a firstlight-emitting subassembly that is configured to emit light into thefirst surface of the user's head when the head-wearable housingstructure is worn on the head of the user; and a first light-sensingsubassembly that is held against the first surface of the user's headand configured to sense first light emitted from the first surface ofthe user's head when the head-wearable housing structure is worn on thehead of the user; a second light-sensing assembly supported by thesecond portion of the head-wearable housing structure, comprising: asecond light-emitting subassembly that is configured to emit light intothe second surface of the user's head when the head-wearable housingstructure is worn on the head of the user; and a second light-sensingsubassembly that is held against the second surface of the user's headand configured to sense second light emitted from the second surface ofthe user's head when the head-wearable housing structure is worn on thehead of the user; and a processor configured to determine a type of ahead gesture of the user based on both the first sensed light and thesecond sensed light, wherein the determined type comprises one of thefollowing: chewing; smiling; frowning; grimacing; gasping; mouthopening; mouth closing; or humming.
 24. The electronic device of claim23, wherein the processor is further configured to adjust afunctionality of the head-wearable electronic device automatically basedon the determined type.
 25. An electronic device comprising: ahead-wearable housing structure, wherein, when the head-wearable housingstructure is worn on a head of a user, the head-wearable housingstructure is configured such that a first portion of the head-wearablehousing structure is held against a first surface of the user's head andsuch that a second portion of the head-wearable housing structure isheld against a second surface of the user's head; a first light-sensingassembly supported by the first portion of the head-wearable housingstructure, comprising: a first light-emitting subassembly that isconfigured to emit light of a first wavelength into the first surface ofthe user's head when the head-wearable housing structure is worn on thehead of the user; and a first light-sensing subassembly that is heldagainst the first surface of the user's head and configured to sensefirst light emitted from the first surface of the user's head when thehead-wearable housing structure is worn on the head of the user, whereinthe sensed first light comprises at least a portion of the light emittedinto the first surface of the user's head by the first light-emittingsubsassembly when the head-wearable housing structure is worn on thehead of the user; a second light-sensing assembly supported by thesecond portion of the head-wearable housing structure, comprising: asecond light-emitting subassembly that is configured to emit light of asecond wavelength different than the first wavelength into the secondsurface of the user's head when the head-wearable housing structure isworn on the head of the user; and a second light-sensing subassemblythat is held against the second surface of the user's head andconfigured to sense second light emitted from the second surface of theuser's head when the head-wearable housing structure is worn on the headof the user, wherein the sensed second light comprises at least aportion of the light emitted into the second surface of the user's headby the second light-emitting subsassembly when the head-wearable housingstructure is worn on the head of the user; and a processor configured todetermine a type of a head gesture of the user based on both the firstsensed light and the second sensed light.
 26. The electronic device ofclaim 25, wherein the determined type comprises one of the following:chewing; smiling; frowning; grimacing; gasping; mouth opening; mouthclosing; or humming.